Apparatus and method for providing information

ABSTRACT

An electronic apparatus includes a display configured to display one or more messages in response to an execution of an application for exchanging the one or more messages between the electronic apparatus and another electronic apparatus; and a processor configured to extract a keyword from the one or more messages for processing a search query by using the keyword and location information of the electronic apparatus.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from U.S. patent application Ser. No.14/813,673, filed Jul. 30, 2015, which claims priority from KoreanPatent Application Nos. 10-2014-0098633, filed on Jul. 31, 2014,10-2015-0022720, filed on Feb. 13, 2015, and 10-2015-0044996, filed onMar. 31, 2015, in the Korean Intellectual Property Office, thedisclosures of which are incorporated herein by reference in theirentireties.

BACKGROUND 1. Field

Apparatuses and methods consistent with exemplary embodiments relate toan apparatus and a method for providing information.

2. Description of the Related Art

Due to a rapid distribution of portable terminals, the use of theportable terminals becomes important in daily life. The portableterminals may provide various functions such as a voice call service, adata transmission service, and various additional services.

In particular, communication between users using smart terminals becomesincreasingly popular. With this trend, various techniques for providinga more convenient communication environment for the communication arebeing developed.

However, existing techniques cannot provide a customized service byusing user's information such as user's location information.

SUMMARY

One or more exemplary embodiments provide a method and an electronicapparatus for providing information of interest of a user by consideringlocation information of the user, and for obtaining information ofinterest of the user.

One or more exemplary embodiments include an electronic device thatgenerates a query that a user may need while the user uses anapplication, performs a search operation, and obtains information ofinterest.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of exemplary embodiments.

According to an aspect of an exemplary embodiment, there is provided anelectronic apparatus including: a display configured to display one ormore messages in response to an execution of an application forexchanging the one or more messages between the electronic apparatus andanother electronic apparatus; and a processor configured to extract akeyword from the one or more messages for processing a search query byusing the keyword and location information of the electronic apparatus.

The display may be further configured to display a graphical userinterface (GUI) configured to receive a user input with respect to thekeyword, and the processor may be further configured to obtaininformation related to a result of the search query, according to theuser input, and the display may be further configured to display theinformation.

The processor may be further configured to obtain the information via atleast one from among a second application that is installed in theelectronic apparatus and an Internet search service.

The processor may be further configured to execute the secondapplication, and to obtain the information by inputting the search queryto the second application.

The display may be further configured to display a graphical userinterface (GUI) configured to receive a user input with respect to thekeyword, and to display a query editing window for editing the searchquery, according to the user input.

The display may be further configured to display the search querydetermined by the processor via the query editing window, and to receiveediting on the search query by a user.

The processor may be further configured to obtain information related toa result of the search query edited according to the received editing.

The processor may be further configured to obtain at least one of thelocation information of the electronic apparatus measured by using aglobal positioning system (GPS) processor included in the electronicapparatus and the location information of the electronic apparatusextracted from the one or more messages.

The processor may be further configured to determine the search query byprioritizing use of the location information extracted from the one ormore messages over use of the location information obtained by using theGPS processor.

The processor may be configured to determine the search query based onuser information associated with the electronic apparatus.

According to an aspect of an exemplary embodiment, there is provided amethod of providing information in an electronic apparatus, the methodincluding: displaying one or more messages in response to execution ofan application for exchanging the one or more messages to between theelectronic apparatus and another electronic apparatus; and extracting akeyword from the one or more messages; and processing a search queryusing the keyword and location information of the electronic apparatus.

The method may further include receiving a user input with respect tothe keyword; and obtaining information related to a result of the searchquery, in response to the user input; and displaying the information.

The obtaining may include obtaining the information via at least onefrom among a second application that is installed in the electronicapparatus and an Internet search service.

The obtaining may include executing the second application, andobtaining the information by inputting the search query to the secondapplication.

The method may further include receiving a user input with respect tothe keyword; and displaying a query editing window for editing thesearch query, according to the user input.

The method may further include displaying the search query via the queryediting window; and receiving editing on the search query by a user.

The obtaining may include obtaining information related to a result ofthe search query edited according to the received editing.

The method may further include obtaining at least one of the locationinformation of the electronic apparatus measured by using a globalpositioning system (GPS) processor included in the electronic apparatusand the location information from the one or more messages.

The determining may include determining the search query by prioritizinguse of the location information extracted from the one or more messagesover use of the location information obtained by using the GPSprocessor.

The determining may include determining the search query based on userinformation associated with the electronic apparatus.

According to an aspect of an exemplary embodiment, there is provided anelectronic apparatus may include: a display configured to display amessage transmitted to or received from another electronic apparatus;and a processor configured to determine a keyword based on the message,and to determine a search query by using the keyword and information ofa user.

The processor may be further configured to determine the keyword byusing a previous message, and a current message that is displayed on thedisplay.

The processor may be further configured to perform a search operation byusing the search query, and to control the display to display a resultof the search operation.

The processor may be further configured to execute the search querycorresponding to the keyword, in response to a user input with respectto the keyword displayed on the display, and to control the display todisplay a result of executing the search query.

The user information may include at least one from among locationinformation of the electronic apparatus of the user, communicationhistory information of the user, and search log information of the user.

The processor may be further configured to display the result ofexecuting the search query on the display together with the message.

The processor may be further configured to control the display todisplay the keyword in an original display state after a lapse of apreset time period.

The display may be further configured to distinguishably display thekeyword in the one or more messages.

The method may further include distinguishably displaying the keyword inthe one or more messages.

The display may be further configured to distinguishably display thekeyword in the message.

According to an aspect of an exemplary embodiment, there is anelectronic apparatus including: a processor configured to extract akeyword from application content that is communicated between theelectronic apparatus and another electronic apparatus via anapplication, generate a search query based on the keyword, and obtaininformation based on a result of the search query; and an outputinterface configured to provide the obtained information to a user whilethe application is executed.

The application may include at least one from among an application forperforming text chatting, an application for performing voice chatting,and an application for performing video chatting.

When at least one of the application for performing voice chatting andthe application for performing video chatting is performed, theprocessor may be further configured to convert a voice signal into textby using a speech to text (STT) engine, and to extract the keyword fromthe converted text.

The processor may include a natural language processor configured torecognize a meaning of words included in the content, and the processormay be further configured to determine whether to extract the keywordbased on the recognized meaning of the words.

The output interface may be further configured to provide the obtainedinformation to the user by using at least one of converting anapplication execution screen, partitioning the application executionscreen, outputting a voice signal, performing a messaging service,performing an e-mail service, performing a social network service (SNS),and performing an application service.

The electronic apparatus may be interoperated with a second electronicapparatus, and the processor may be further configured to extract thekeyword from content that is communicated between the second electronicapparatus and the another electronic apparatus by executing theapplication in the second electronic apparatus, generate the searchquery based on the keyword, and obtain the information based on a resultof executing the search query; and the output interface is configured toprovide the obtained information to the user through the electronicapparatus.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and/or other aspects will become more apparent by describingcertain exemplary embodiments with reference to the accompanyingdrawings in which:

FIG. 1A illustrates a user interface (UI) of an electronic device,according to an exemplary embodiment;

FIG. 1B illustrates a UI of an electronic device, according to anotherexemplary embodiment;

FIG. 2A illustrates a flowchart of a method, performed by an electronicdevice, of providing information, according to an exemplary embodiment;

FIG. 2B illustrates a flowchart of a method, performed by an electronicdevice, of providing information, according to another exemplaryembodiment;

FIG. 2C illustrates a flowchart of a method, performed by an electronicdevice, of providing information, according to still another exemplaryembodiment;

FIGS. 3, 4A, 4B, 4C, 5, 6A, 6B, 6C, and 7 illustrate detailed UIenvironments provided during communication via a messaging service;

FIGS. 8, 9A, 9B, and 10 illustrate detailed UI environments that areprovided during communication via a voice chatting service;

FIGS. 11 through 15 illustrate detailed UI environments that areprovided during communication via a mailing service;

FIGS. 16 through 19 illustrate detailed UI environments that areprovided during communication via a social network service (SNS);

FIG. 20 illustrates a detailed UI environment that is provided while ascheduler service is used;

FIG. 21 illustrates a detailed UI environment that is provided while aword processor application is used;

FIGS. 22A through 22I illustrate examples in which, when an electronicdevice is a smartphone or a tablet personal computer (PC) and a userperforms text chatting by using the smartphone or the tablet PC,information obtained by the electronic device is provided to the uservia an output interface;

FIGS. 23A through 23E illustrate examples in which, when an electronicdevice is a smartphone or a tablet PC and a user performs voice chattingby using the smartphone or the tablet PC, information obtained by theelectronic device is provided to the user via an output interface;

FIGS. 24A through 24F illustrate examples in which, when an electronicdevice is a smartwatch and a user performs text chatting by using thesmartwatch, information obtained by the electronic device is provided tothe user via an output interface;

FIGS. 25A and 25B illustrate examples in which, when an electronicdevice is a smartwatch, and a user performs text chatting by using thesmartwatch and a smartphone that is an electronic device interoperatingwith the smartwatch, information obtained by the electronic device isprovided to the user via an output interface;

FIGS. 26A and 26B illustrate examples in which, when an electronicdevice is a smartwatch and a user performs voice chatting by using thesmartwatch, information obtained by the electronic device is provided tothe user via an output interface;

FIGS. 27A and 27B illustrate still other examples in which, when anelectronic device is smartglasses and a user performs text chatting byusing the smartglasses, information obtained by the electronic device isprovided to the user via an output interface;

FIGS. 28A and 28B illustrate examples in which, when an electronicdevice is smartglasses and a user performs voice chatting by using thesmartglasses, information obtained by the electronic device is providedto the user via an output interface;

FIGS. 29A and 29B illustrate examples in which, when an electronicdevice includes various electronic devices, and a user communicates withanother user via at least one of voice chatting and text chatting byusing the electronic device, information obtained by the electronicdevice is provided to the user via an output interface;

FIGS. 30A through 30G, 31A and 31B illustrate settings UIs that areprovided by an electronic device, according to exemplary embodiments;

FIGS. 32 through 35 are flowcharts illustrating a method, performed byan electronic device, of providing information, according to exemplaryembodiments;

FIG. 36 illustrates an electronic device, according to an exemplaryembodiment;

FIG. 37 is a block diagram illustrating an electronic device, accordingto another exemplary embodiment;

FIG. 38 illustrates a configuration of a natural language processorshown in FIGS. 36 and 37;

FIG. 39 is a block diagram illustrating an electronic device, accordingto still another exemplary embodiment;

FIG. 40 is a block diagram illustrating an operation, performed an theelectronic device, of recognizing an entity, according to an exemplaryembodiment;

FIG. 41 illustrates a table chart generated by a named-entity recognizerafter the named-entity recognizer recognizes entities, according to anexemplary embodiment;

FIG. 42 is a block diagram illustrating an operation, performed by anelectronic device, of obtaining a user input, according to an exemplaryembodiment;

FIG. 43 is a block diagram illustrating an operation, performed by anelectronic device, of obtaining a user input, according to anotherexemplary embodiment;

FIG. 44 is a block diagram illustrating a configuration of a userterminal device, according to an exemplary embodiment;

FIG. 45A is a block diagram illustrating a software configuration of anuser terminal device, according to an exemplary embodiment;

FIG. 45B illustrates a user settings screen provided by an electronicdevice, according to an exemplary embodiment; and

FIG. 46 illustrates a UI of an electronic device, according to anotherexemplary embodiment.

FIGS. 47 through 50 illustrate examples of a UI provided by theelectronic device 100 for a communication via a messaging service.

DETAILED DESCRIPTION

Certain exemplary embodiments are described in greater detail below withreference to the accompanying drawings, wherein like reference numeralsrefer to like elements throughout. In this regard, the exemplaryembodiments may have different forms and should not be construed asbeing limited to the descriptions set forth herein. Accordingly, theexemplary embodiments are merely described below, by referring to thefigures, to explain aspects of the present description. Sizes ofelements in the drawings may be exaggerated for convenience ofexplanation. In other words, since sizes and thicknesses of componentsin the drawings are arbitrarily illustrated for convenience ofexplanation, the following exemplary embodiments are not limitedthereto.

All terms including descriptive or technical terms which are used hereinshould be construed as having meanings that are obvious to one ofordinary skill in the art. However, the terms may have differentmeanings according to an intention of one of ordinary skill in the art,precedent cases, or the appearance of new technologies. Also, some termsmay be arbitrarily selected by the applicant, and in this case, themeaning of the selected terms will be described in detail in thedetailed description of the disclosure. Thus, the terms used herein haveto be defined based on the meaning of the terms together with thedescription throughout the specification.

Throughout the specification, when a part “includes” or “comprises” anelement, unless there is a particular description contrary thereto, thepart can further include other elements, not excluding the otherelements. Throughout the specification, it will also be understood thatwhen an element is referred to as being “connected to” or “coupled with”another element, it can be directly connected to or coupled with theother element, or it can be electrically connected to or coupled withthe other element by having an intervening element interposedtherebetween. In the following description, terms such as “unit”,“module” or the like indicate a unit for processing at least onefunction or operation, wherein the unit and the module may be embodiedas hardware or software or embodied by combining hardware and software.

Throughout the specification, a term “user” may mean a user of anelectronic device. Throughout the specification, a communication servicemay mean a one-to-one service, a one-to-many service, or a many-to-manyservice that facilitates an exchange of information between users, e.g.,conversation between users.

Throughout the specification, an application indicates a set of computerprograms designed to perform particular tasks. Throughout thespecification, the application may vary. For example, the applicationmay include, but is not limited to, a game application, a videoreproducing application, a map application, a memo application, acalendar application, a phone-book application, a broadcastingapplication, an exercise support application, a payment application, aphoto folder application, etc. The application may also be called an“App”.

The inventive concept will now be described more fully with reference tothe accompanying drawings, in which exemplary embodiments are shown. Theinventive concept may, however, be embodied in many different forms andshould not be construed as being limited to the exemplary embodimentsset forth herein; rather, these exemplary embodiments are provided sothat this disclosure will be thorough and complete, and will fullyconvey the inventive concept to those of ordinary skill in the art. Inthe following description, well-known functions or constructions are notdescribed in detail since they would obscure the disclosure withunnecessary detail. Throughout the specification, like referencenumerals in the drawings denote like elements.

As used herein, the term “and/or” includes any and all combinations ofone or more of the associated listed items. Expressions such as “atleast one of,” when preceding a list of elements, modify the entire listof elements and do not modify the individual elements of the list.

Throughout the specification, a message may mean a unit text set or aunit voice set including at least one from among a word, a phrase, and aclause that may be a part of a conversation between users.

Throughout the specification, a keyword may be included in a message andmay mean a word, a phrase, and/or a clause related to a meaning of themessage obtained by performing natural language analysis on the message.

Throughout the specification, a query may mean a word, a phrase, and/orclause that is an object for searching for information that matches witha query condition.

FIG. 1A illustrates a user interface (UI) of an electronic device 100,according to an exemplary embodiment.

The electronic device 100 may be embodied as a smartphone, a tabletpersonal computer (PC), a wearable device, a personal digital assistant(PDA), a laptop computer, a cellular phone, a mobile phone, anenterprise digital assistant (EDA), a portable multimedia player (PMP),a personal navigation device (PND), a portable navigation device (PND),a handheld game console, a mobile internet device (MID), or anelectronic book (e-Book).

Throughout the specification, the wearable device may include ahead-mounted display (HMD) that is worn on a head. For example, the HMDmay include, but is not limited to, glasses, a helmet, a hat, or thelike. Throughout the specification, the wearable device may include aring, a necklace, a bracelet, shoes, earrings, a hair band, cloth,gloves, a thimble, or the like.

The electronic device 100 according to an exemplary embodiment maydisplay messages 5, 6, 7, and 8 on its screen via a communicationservice. Throughout the specification, a message may mean a unit set oftexts or a unit set of voices, which is a part of a conversation betweenusers and is exchanged as a set of sequence between the electronicdevice 100 and another electronic device. For example, referring to FIG.1A, a user (Chris) transmits the message 7 ‘What do you want to do?’ toHyunjin at 08:28 A.M. on Aug. 6, 2013. Also, for example, referring toFIG. 1A, Hyunjin transmitted the message 8 ‘I want to eat pizza.’ at08:30 A.M. on Aug. 6, 2013. For example, in FIG. 1A, the text that isincluded in each speech balloon in a conversation between Chris andHyunjin may be defined as a message.

The electronic device 100 according to an exemplary embodiment maydetermine whether to extract a keyword from the message. The electronicdevice 100 according to an exemplary embodiment may extract at least onekeyword from a message displayed on a screen via a communicationservice. For example, the electronic device 100 may extract at least onekeyword from text and image in the message. For example, the electronicdevice 100 may extract at least one keyword from a multimedia fileattached in the message. The multimedia file may include a picture file,a video file and a sound file. The keyword may be included in a messageor may be related to contents of the message, and may include at leastone of a word, a phrase, and a clause related to a meaning of themessage. For example, the electronic device 100 may extract ‘pizza’ as akeyword from the message 8.

The electronic device 100 according to an exemplary embodiment maydisplay the extracted keyword. Referring to FIG. 1A, the electronicdevice 100 marks the keyword by using an underline. Alternatively, theelectronic device 100 may differentiate the keyword from other contentsof the message 8 by changing a font, by changing a font size, byhighlighting the keyword, by performing bold lettering on the keyword,by flickering the keyword, by shading the keyword, or by shading aspeech balloon.

The electronic device 100 according to an exemplary embodiment maygenerate a query related to the keyword extracted from the message, andmay obtain information based on the query and location information ofthe electronic device 100. The query may mean at least one of a word, aphrase, and a clause that is an object for searching for informationthat matches with a query condition. The query may be related to thekeyword. The query may be generated based on at least one keyword.

The electronic device 100 according to another exemplary embodiment maygenerate a query, based on a keyword and location information of theelectronic device 100. For example, the electronic device 100 may obtaininformation via the query generated based on the keyword and thelocation information of the electronic device 100, and may display theinformation.

The electronic device 100 may obtain information in the electronicdevice 100 or may obtain information retrieved from a server. Theelectronic device 100 may obtain information from a search enginewebsite.

Also, the electronic device 100 may provide information obtained by aprocessor of the electronic device 100. In an exemplary embodiment, theinformation is visually or acoustically provided to a user to berecognized. For example, when a user touches a keyword that isdifferently marked from the rest of contents of the message 8, theelectronic device 100 may provide the obtained information.

In an exemplary embodiment, the processor of the electronic device 100may obtain, regardless of a request from the user, information relatedto the keyword that is differently marked from the rest of contents ofthe message 8, and may store the information in a memory (not shown) ofthe electronic device 100. Also, when the user touches the differentlymarked (e.g., highlighted) keyword, the electronic device 100 mayoutput, from the memory of the electronic device 100, the informationthat is obtained regardless of the request from the user and stored inthe memory of the electronic device 100, and may provide the informationto the user.

In an exemplary embodiment, when the user touches the differently marked(e.g., highlighted) keyword, the processor of the electronic device 100may obtain the information related to the keyword and may provide theinformation to the user without storing the information in the memory ofthe electronic device 100.

A method of extracting the keyword, a method of generating the querybased on the extracted keyword, and a method of obtaining theinformation based on the query and the location information of theelectronic device 100, performed by the electronic device 100, will bedescribed in detail later with reference to FIG. 2A.

FIG. 1B illustrates a UI of the electronic device 100, according toanother exemplary embodiment.

The electronic device 100 according to an exemplary embodiment mayprovide a query generated by a processor of the electronic device 100.The electronic device 100 according to an exemplary embodiment mayreceive a query edited by a user. For example, the user may compose asecond query by editing a first query generated by the processor, andmay input the second query to a query editing window 1 provided via agraphical user interface (GUI) of the electronic device 100. The usermay input, as the second query, the first query, which is generated bythe processor, to the electronic device 100. The electronic device 100may obtain information, based on the edited second query, and may obtaininformation, based on the edited second query and the locationinformation of the electronic device 100.

For example, as illustrated, the electronic device 100 may provide thequery editing window 1, and may provide a word ‘pizza’ as a defaultvalue of a query. Also, the electronic device 100 may receive the queryedited by the user, may obtain information by using various methods, andmay provide the obtained information to the user. Detailed descriptionsabout operations of the electronic device 100 are provided in detailwith reference to FIGS. 2A and 2B.

Hereinafter, the operations of the electronic device 100 are described.

FIG. 2A illustrates a flowchart of a method of providing information,the method being performed by the electronic device 100 and described incorrespondence with FIG. 1A, according to an exemplary embodiment.

Referring to FIG. 2A, a processor of the electronic device 100 mayextract a keyword from a message that is displayed on a screen of theelectronic device 100 via a communication service (S110). A method usedby the processor to extract the keyword may include semantic analysisand/or statistical analysis as shown in Table 1.

TABLE 1 Method of extracting Base of keyword analysis Detailed examplesof base of analysis semantic content of content extracted from aconversation analysis conversation between a user and anothercommunication between users party, conversation content about a locationof the user, conversation content about a mental state of the user,conversation content about a past experience of the user statistical logof user's frequency of the user passing a place where analysis locationthe user is currently located or of the user passing a particular place,the number of times that the user passed a place where the user iscurrently located or the number of times that the user passed aparticular place, a total time during which the user stays in a placewhere the user is currently located or in a particular place, a totaltime during which the user stays in a place where the user is currentlylocated or in a particular place during a particular time period, a timewhen the user first visits a current place or a particular place userinput schedule of the user, address of the user, information user'spreference for a pizza restaurant, user's tendency in selecting a menuhistory history information about the user including information acurrent location of the user, a payment about the user history of theuser, an accumulated location history of the user, a call history of theuser priority orders content of conversation between users, of referencelocation log of the user, user input contents or information, historyinformation about the priority orders user of results of semanticanalysis

[1. Extraction of Keyword]

For example, as illustrated in FIGS. 1A and 1B, a message displayed onthe screen may be ‘What do you want to do?’ (a first message 7) and ‘Iwant to eat pizza’ (a second message 8). The processor may analyzemeanings of the first and second messages 7 and 8, and may extract‘pizza’ as a keyword from the second message 8.

The processor of the electronic device 100 may perform the semanticanalysis and/or the statistical analysis on the message, and thus mayextract ‘pizza’ as the keyword. The processor of the electronic device100 may extract the keyword by considering a situation related to themessage. For example, the processor of the electronic device 100 mayextract the keyword, based on a type of words included in the message, arelation between the words, and meanings of the words.

The processor may recognize entities in messages by using a methoddescribed with reference to FIGS. 36 through 41, and may extract akeyword by considering the number of times each of the entities areused. The method, performed by the processor, of recognizing theentities will be described later.

The processor of the electronic device 100 may control variousprocessing operations of the electronic device 100. For example, theprocessor of the electronic device 100 may include a processor such as acentral processor unit (CPU), a micro controller unit (MCU), or a microprocessor unit (MPU).

[1-1. Semantic Analysis]

The processor of the electronic device 100 may extract the keyword byperforming the semantic analysis.

For example, the processor may extract ‘pizza’ as the keyword by furtherconsidering content of conversation between users and user-relatedinformation. Here, the user-related information may include at least oneof a location log of the user, user's previously input information, andthe history information about the user. The user-related information maymean information about only one user or information about at least twousers.

The processor may extract the keyword, based on at least one of, forexample but not limited to, 1) the content of conversation between theusers, 2) a user's location log, 3) the user's previously inputinformation, and 4) the history information about the user.

The processor may perform natural language analysis on the content ofconversation between the users, may recognize the content ofconversation between the users, and may extract the keyword. Forexample, in the exemplary embodiment of FIG. 1A, the processor mayextract the keyword, based on content extracted from a conversationbetween the user and another communication party, conversation contentabout a location of the user, conversation content about a mental state(e.g., a feeling) of the user, and/or conversation content about a pastexperience of the user.

For example, the processor may perform extraction, based on theconversation content, and in the exemplary embodiment of FIG. 1A, theprocessor may extract ‘pizza’ as ‘the keyword from the message 8. Theprocessor may recognize a meaning of the message included in theconversation between the users, and may predict a type of informationthat is desired by the user, based on the meaning of the message. Forexample, the processor may analyze the meaning of the message as ‘theuser wants to eat pizza’, and may predict, based on the meaning, thatthe user currently wants to find a pizza restaurant near a currentlocation of the user (a first situation) or that the user needs atelephone number of the pizza restaurant near the current location ofthe user (a second situation). Here, a situation of the user mayindicate environmental elements related to the user.

Also, when the processor extracts the keyword, based on the meaning ofthe message, the processor may consider the user's location log.

For example, the user's location log may include a frequency of the userpassing a place where the user is currently located or of the userpassing a particular place, the number of times that the user passed aplace where the user is currently located or that the user passed aparticular place, a total time during which the user stays in a placewhere the user is currently located or in a particular place, a totaltime during which the user stays in a place where the user is currentlylocated or in a particular place during a particular time period, and/ora time when the user first visits a current place or a particular place.

The electronic device 100 may check location information of the user,and may use the location information to extract the keyword. Thelocation information of the user may be continuously checked inreal-time, and may be stored in the electronic device 100.

The location information of the user may be measured by locationinformation of the electronic device 100. For example, the locationinformation of the electronic device 100 may include a globalpositioning system (GPS)-coordinates value, identification (ID) of abase station, a service set identifier (SSID) and a network address ofan access point (AP). The location information of the user may bemeasured by a measuring method using various wireless communicationtechnologies including, for example, a diffuse-infrared ray, ultrasound,Bluetooth, ultra-wideband (UWB), radio-frequency identification (RFID),Wi-Fi protected setup (WPS), or the like.

The location information may indicate log information about a locationof the user or the electronic device 100 with respect to a time. Forexample, the electronic device 100 may include a GPS module. A method,performed by the GPS module, of measuring the location of the electronicdevice 100, will be described in detail with reference to FIG. 45.

Also, when the processor extracts the keyword, based on the meaning ofthe message, the processor may consider information that is previouslyinput by the user.

For example, the user's previously input information may include anaddress of the user, a user's preference for a pizza restaurant, auser's tendency in selecting a menu, and/or an estimated schedule of theuser.

For example, when the address of the user is relatively far away fromthe current location, and a total time period in which the user stays ina current place is less than a reference time (e.g., one hour), theprocessor may recognize that the current location of the user is not afamiliar place to the user. Based on the analysis, the processor maypredict that the user currently wants to find the pizza restaurant nearthe current location of the user (the first situation) or that the userneeds the telephone number of the pizza restaurant near the currentlocation of the user (the second situation).

Also, when the processor predicts the keyword based on the meaning ofthe message, the processor may consider the history information aboutthe user.

For example, the history information about the user may indicate ahistory about the user which is recorded in the electronic device 100and a server. Also, the history about the user may include a historyrecorded by the user when the user uses the electronic device 100, and ahistory recorded to the server, by the user, when the user uses an Appservice. Also, the history about the user may include content that isnot directly recorded by the user but is indirectly recorded when theuser uses the electronic device 100 or the server. For example, thehistory information about the user may include user-related informationsuch as the current location of the user, a payment history of the user,an accumulated history of the user, a call history of the user, or thelike.

[1-2. Statistical Analysis]

The processor of the electronic device 100 may extract the keyword viathe statistical analysis.

The processor may determine priority orders of reference contents. Forexample, when the processor performs the semantic analysis, based on thecontent of conversation between the users, the user's location log, theuser input information, and the history information about the user, theprocessor may determine a base of analysis to be preferentiallyconsidered from among bases of analyses to extract the keyword. Forexample, when a location of the electronic device 100 measured by theGPS module of the electronic device 100 is different from locationinformation mentioned in the content of conversation between the users,the processor may extract the keyword by preferentially considering thelocation information mentioned in the content of conversation betweenthe users.

Also, the processor may determine priority orders of predictedsituations of the user. For example, the processor may determine thatone situation has a higher possibility of occurrence, wherein thesituation is one of the first situation in which the user currentlywants to find the pizza restaurant near the current location of the userand the second situation in which the user needs the telephone number ofthe pizza restaurant near the current location of the user in theexemplary embodiments of FIGS. 1A and 1B.

According to accumulated location information of the user, if the userfrequently visited a particular pizza restaurant, the processor maydetermine, as a first priority, the first situation in which the usercurrently wants to find the pizza restaurant near the current locationof the user.

Alternatively, for example, when a place where the user is currentlylocated is outdoor, and according to accumulated location information(or accumulated location history) of the user, if the user moves over apredetermined time (e.g., 30 minutes), or if going out is recorded to ascheduler of the user, the processor may determine, as the firstpriority, the first situation in which the user currently wants to findthe pizza restaurant near the current location of the user.

For example, when the number of times that the user calls a pizzarestaurant is equal to or greater than a preset number of times in acall history of the user, the processor may determine, as the firstpriority, the second situation in which the user needs the telephonenumber of the pizza restaurant near the current location of the user.

Alternatively, for example, when a place where the user is currentlylocated is a house of a friend of the user, and according to thescheduler of the user, the user intends to stay in the place over apredetermined time (e.g. 3 hours), the processor may determine, as thefirst priority, the second situation in which the user needs thetelephone number of the pizza restaurant near the current location ofthe user.

Although it is described that the processor determines the firstsituation or the second situation in the above example, however, theexemplary embodiments are not limited thereto. The processor of theelectronic device 100 may determine one of any number of differentsituations based on the meaning of the message and the historyinformation of the user.

The processor may extract the keyword ‘pizza’ by performing theaforementioned statistical analysis and/or the semantic analysis.

[1-3. Limitation in Extraction of Keyword]

The processor of the electronic device 100 may set a keyword, inconsideration of the number of times of occurrence of a correspondingword. For example, the processor may set, as the keyword, a word that isrepeatedly mentioned over a predetermined number of times, by the user,in a conversation. For example, when the user mentions the word at leastfive times or with at least about 20% frequency of use in theconversation, the processor may set the word as the keyword.

For example, the processor may perform named-entity recognition, and maymeasure a number of times of repetition or a repetition rate (or arepetition frequency) of each of entities. For example, the processormeasures a number of times of repetition or a repetition rate of each ofentities that are recognized via an external server.

A method of recognizing an entity via the processor or a server will bedescribed later.

Also, the external server may perform named-entity recognition, and maycalculate a number of times of repetition or a repetition rate of eachof the entities, and the processor may receive data about the number oftimes of repetition or the repetition rate of each of the entities andmay refer to the data to set a keyword.

For example, the processor may not set, as the keyword, an entity thatis mentioned less than a preset number of times. Also, for example, theprocessor may not set, as the keyword, an entity that is mentioned lessthan a preset rate. The aforementioned algorithm for limiting extractionof the keyword may also be applied to an algorithm for generating aquery.

The processor may mark the extracted keyword differently from otherwords in the message. The processor may receive a user input ofselecting the extracted keyword, and may perform operations S130 throughS190. In another exemplary embodiment, the processor may performoperations S130 through S190 without receiving a user input.

Also, the processor may limit the extraction of the keyword by using themethod of semantic analysis as described above. In other words, theelectronic device 100 may recognize entities, based on 1) the content ofconversation between the users, 2) the user's location log, 3) theuser's previously input information, and 4) the history informationabout the user, and may limit recognition of the entities.

For example, when the electronic device 100 receives a message “Pleasecome to a funeral hall next to a pizza restaurant”, the processor, byperforming the semantic analysis, may recognize <the funeral hall nextto the pizza restaurant> as an entity, or may recognize <the pizzarestaurant> as an entity but may not extract the recognized entity asthe keyword.

For example, the processor may recognize that a connective word (e.g.,next to, near, or rear) for connecting words expressing places is usedin <the funeral hall next to the pizza restaurant>, and may recognize<the funeral hall next to the pizza restaurant> as an entity.

[1-4. Extraction of Keyword by User Input]

The processor may determine a keyword by receiving a user input. A usermay select the keyword from messages of a conversation. The user mayselect the keyword in a preset manner. For example, the user may selectthe keyword in the manner that is preset via settings. For example, theuser may select the keyword in a long press touch and drag operation.

The processor may generate and provide a query related to the keyword(S130). A method used by the processor to generate the query is semanticanalysis and/or statistical analysis as shown in Table 2.

TABLE 2 Method of determining a query Base of analysis Semantic analysisContent of conversation between users User's location log User'spreviously input information History information about the userStatistical analysis Priority orders of reference contents or priorityorders of results of the semantic analysis

[2. Generation of Query]

As shown in the exemplary embodiment of FIG. 1B, the processor mayperform the semantic analysis and/or the statistical analysis, based onthe extracted keyword, and may generate ‘good day pizza’ as the query.For example, the processor may generate ‘good day pizza’ as the query,based on at least one of 1) the content of conversation between theusers, 2) the user's location log, 3) the user's previously inputinformation, and 4) the history information about the user.

The processor may analyze the content of conversation between the users,and may generate ‘good day pizza’ as the query. For example, if theprocessor semantically recognizes, from the content of conversationbetween the users, that the user wants to eat at a restaurant ‘good daypizza’, the processor may generate ‘good day pizza’ as the query.

The processor may analyze the user's location log, and may generate‘good day pizza’ as the query. The processor may analyze the user'slocation log, and may provide the query that matches with a situation ofthe user.

For example, if the processor recognizes that the user wants to find apizza restaurant near a current location, and a record showing that theuser visited the restaurant ‘good day pizza’ occurs frequently in anaccumulated history of a user's location, the processor may generate‘location of good day pizza’ as the query.

For example, if the processor recognizes that the user wants to have atelephone number of the pizza restaurant near the current location, anda record showing that the user called the restaurant ‘good day pizza’occurs frequently in a call history of the user, the processor maygenerate ‘telephone number of good day pizza’ as the query.

For example, if the processor recognizes that the user wants to find thepizza restaurant near the current location or to have the telephonenumber of the pizza restaurant near the current location, and a recordshowing that the user paid a bill in the restaurant ‘good day pizza’occurs frequently in payment detail reception messages in a history ofthe user, the processor may generate ‘good day pizza’ as the query.

The processor may analyze the user's previously input information, andmay generate ‘good day pizza’ as the query. The processor may recognize,from the information input to the electronic device 100, that therestaurant ‘good day pizza’ is a user's favorite pizza restaurant.

The user may directly record the restaurant ‘good day pizza’ as theuser's favorite pizza restaurant to the electronic device 100. Also, theuser may record a plurality of pieces of information about therestaurant ‘good day pizza’ to the electronic device 100. In this case,the processor may recognize, by performing the statistical analysis,that the restaurant ‘good day pizza’ is the user's favorite pizzarestaurant.

The processor may recognize, from information input to a server via anapplication service, that the restaurant ‘good day pizza’ is the user'sfavorite pizza restaurant. For example, by referring to a comment thatis written by the user and is input to the server via a social networkservice (SNS), the processor may determine that the user prefers therestaurant ‘good day pizza’.

The user may directly record the restaurant ‘good day pizza’ as theuser's favorite pizza restaurant to a database of the server via anapplication service. Also, the user may record a plurality of pieces ofinformation about the restaurant ‘good day pizza’ to the database of theserver via the application service. In this case, the processor mayrecognize, by using a statistical method, that the restaurant ‘good daypizza’ is the user's favorite pizza restaurant.

As described above, the processor may perform the semantic analysisand/or the statistical analysis, and thus may extract ‘good day pizza’as the query.

The electronic device 100 may receive an information provision requestfrom the user, and may extract the query. For example, a display 9(refer to FIG. 1A) may receive a user input (e.g., a touch) of selectinga portion corresponding to a keyword that is displayed on a screen, andmay extract a query. For example, before the electronic device 100receives the information provision request from the user, the electronicdevice 100 may determine a keyword, and may display the keyword bymarking the keyword differently from other contents, and after theelectronic device 100 receives the information provision request fromthe user, the electronic device 100 may extract the keyword.

Throughout the specification, the information provision request mayinclude an input by the user to request the electronic device 100 toprovide information. For example, the user may input the request byusing various input methods (e.g., a touch, a click, a double click,etc.) to request the electronic device 100 to provide the information,and the request by the user may be referred to as the informationprovision request. Also, the user may input a particular commandincluding a voice and/or a text to request the electronic device 100 toprovide the information, and the request by the user may be referred toas the information provision request or an information provision requestinput.

The aforementioned algorithm for limiting the extraction of the keywordmay also be applied to the algorithm for generating the query.

The processor may obtain the information, based on the query and thelocation information of the electronic device 100 (S170). A method usedby the processor to generate the information is as shown in Table 3.

TABLE 3 Method of obtaining information Receive from an external sourceObtain information via an App service of an electronic device (e.g.,Google map) Obtain information via an Internet search service (e.g.,Yahoo) Obtain from an internal source Obtain, from an internal storagedevice, of the electronic device image files, a telephone number list,an address list, a recently-retrieved information list, etc.

[3. Obtainment of Information]

The electronic device 100 may perform various types of informationretrieving operation, based on the query and the location information ofthe electronic device 100, and thus may obtain the information. Theinformation obtained by the electronic device 100 may include atwo-dimensional (2D) image, a three-dimensional (3D) image, a 2D movingpicture, a 3D moving picture, texts generated by using variouslanguages, contents of various fields, and applications that providevarious services.

After the electronic device 100 receives a user input of selecting akeyword, the electronic device 100 may perform an operation of obtaininginformation corresponding to the selected keyword.

Also, before the electronic device 100 receives the user input ofselecting the keyword, the electronic device 100 may perform theoperation of obtaining the information corresponding to the selectedkeyword, and may store the obtained information in a storage of theelectronic device 100. The electronic device 100 may receive the userinput of selecting the keyword, and may provide the information storedin the storage to the user.

Hereinafter, the operation of obtaining the information is described indetail.

[3-1. Obtainment of Information Via App Service]

The electronic device 100 may obtain related information from anexternal search server.

For example, the electronic device 100 may obtain information via an Appservice.

The electronic device 100 may determine a service according to a currentstate of an App, may perform various types of information retrievingoperation, based on the query and the location information of theelectronic device 100, and thus may obtain the information.

For example, according to the location information of the user, when theuser is determined to be located in South Korea, the electronic device100 may select a map application service provided by a South Koreanservice provider, and when the user is located in Japan, the electronicdevice 100 may select a map application service provided by a Japaneseservice provider.

For example, the electronic device 100 may refer to the number of timesthat the user uses map applications, and may select a map applicationservice frequently used by the user of the electronic device 100.

The electronic device 100 may request the user to select an App service,may receive a response from the user, and may select the App service,based on the response from the user.

For example, the electronic device 100 may select candidate Apps (e.g.,first through fifth Apps) to be used, in consideration of a user'ssituation, may request the user to select at least one of the firstthrough fifth Apps, and when the user selects the first App, theelectronic device 100 may retrieve information by using the first App.

According to an exemplary embodiment, the electronic device 100 mayselect, from among user-selected App services, an App service thatmatches with a user's current situation.

For example, the electronic device 100 may select, from amonguser-selected first through fifth Apps, the first App according to acurrent situation of the user. For example, if the user previouslyselected first through fifth restaurant Apps, the electronic device 100may select a ‘good day pizza’ restaurant App, according to a currentsituation of the user.

According to an exemplary embodiment, the electronic device 100 mayrequest the user to select at least one of the user-selected Appservices, may receive a response from the user, and may select an Appservice according to the response from the user.

For example, the electronic device 100 may request the user to select atleast one of the user-selected first through fifth Apps, and when theuser selects the first App, the electronic device 100 may retrieveinformation by using the first App.

[3-2. Obtainment of Information Via Internet Search Service]

The electronic device 100 may obtain related information from anexternal search server.

For example, the electronic device 100 may obtain information via aninternet search service.

The electronic device 100 may determine the internet search service, mayperform various types of information retrieving, based on a query andlocation information of the electronic device 100, and thus may obtainthe information. The query may indicate an automatically extracted queryor a query that is edited by a user.

For example, the internet search service may indicate a commercializedsearch service including Yahoo, Google, Bing, Naver, etc. For example,the internet search service may indicate a search service such as alibrary database of a university, a thesis search engine website, or adatabase of a research institute, which may be accessed by authorizedusers.

The electronic device 100 may obtain the information, based on the queryand the location information of the electronic device 100, and mayprovide the information to the user.

Therefore, the electronic device 100 according to an exemplaryembodiment may allow the user to conveniently receive informationrelated to ongoing a communication service while the user uses thecommunication service.

[3-3. Obtainment of Information in Electronic Device]

The electronic device 100 may obtain information of interest from aninternal storage device. Hereinafter, the information of interest isreferred to as the interest information.

For example, the electronic device 100 may obtain the interestinformation from image files, a telephone number list, an address list,and a recently-retrieved information list that are stored in theinternal storage device. When the electronic device 100 is a wearabledevice, the electronic device 100 may obtain the interest informationfrom a storage device included in another electronic device that isconnected with the electronic device 100 via short-distancecommunication such as Bluetooth.

Therefore, the electronic device 100 according to an exemplaryembodiment may allow the user to conveniently receive informationrelated to a communication service while the user uses the communicationservice.

An output interface may provide the information obtained by theprocessor (S190). A method used by the processor to provide theinformation is as shown in Table 4.

TABLE 4 Type of electronic Communication Information providing devicemethod method Smartphone Text chatting, voice Screen conversion, screen(or tablet PC) chatting, video partition, voice, voice and chattingscreen partition, voice and screen conversion, a messaging service, ane-mail service, an SNS service, an application service, etc. Wearabledevice Text chatting, voice Provision of summarized (smartwatch, HMD)chatting, video primary information (an chatting image, a text, voice,etc.) Smartphone Text chatting, voice A wearable device provides (ortablet PC) chatting, video summarized information, and interoperatingwith chatting a smartphone a wearable device (or a tablet PC) providesdetailed information

[4. Provision of Information]

The electronic device 100 may provide information, which is obtainedbased on a query and location information of the electronic device 100,to a user in various manners.

For example, the electronic device 100 may provide the obtainedinformation to the user via at least one of voice, an image, and a text.A method of providing the obtained information, the method beingperformed by the electronic device 100, may vary according to types ofthe electronic device 100. The method, performed by the electronicdevice 100, of providing the obtained information, will be described indetail with reference to FIGS. 22 through 29.

FIG. 2B illustrates a flowchart of a method of providing information,the method being performed by the electronic device 100 and described incorrespondence with FIG. 1B, according to another exemplary embodiment.

The electronic device 100 may extract a keyword from a message used incommunication, may generate a first query related to the keyword, andmay provide the first query to a user (S110_a and S130_a). Theelectronic device 100 may provide the first query by using variousmethods. For example, the electronic device 100 may provide a query bygenerating a speech balloon, by using a pop-up window, or by performingscreen conversion.

The electronic device 100 may receive a second query edited by the user(S150_a).

[5. Editing Query]

The electronic device 100 may receive the edited second query from theuser (S150_a). The electronic device 100 may allow the user to edit aquery in a manner such that the query is edited in a text, a gesture,voice, or a sketch.

When the first query generated by the electronic device 100 is providedto the user, the user may edit the query by reflecting his/her exactintention to the query. Throughout the specification, the term ‘editedquery’ may mean the query that is edited by the user.

The user may edit a query provided by the electronic device 100 via anoutput interface. For example, the user may edit the query provided bythe electronic device 100 and may input the edited query to theelectronic device 100. For example, the user may not change the queryprovided by the electronic device 100 and may input the query asprovided. The electronic device 100 may receive the edited query ornon-edited query that is input by the user.

The electronic device 100 may obtain information, based on the editedquery and location information of the electronic device 100 (S170_a).The electronic device 100 may provide the obtained information invarious manners (S190_a). The method of providing the obtainedinformation, the method being performed by the electronic device 100,will be described in detail with reference to FIGS. 22 through 29.

FIG. 2C illustrates a flowchart of a method, performed by the electronicdevice 100, of providing information, according to still anotherexemplary embodiment.

The electronic device 100 may extract a keyword from a message used incommunication, may generate a query, based on the keyword and locationinformation, and may provide the generated query to a user (S110_b andS130_b).

The electronic device 100 may recognize the location information invarious manners, and may provide the query. For example, the electronicdevice 100 may generate the query by considering location information ofthe electronic device 100 that is obtained a GPS module of theelectronic device 100, or may generate the query by considering locationinformation obtained according to a word included in the message,connection relation between words (e.g., a connective word (e.g., nextto, near, or rear) for connecting words expressing places), meaning ofthe word, etc.

The electronic device 100 may use the location information obtained inthe aforementioned manners to generate a keyword and/or the query. Also,the electronic device 100 may use the location information to obtaininformation to be provided to the user.

The electronic device 100 may obtain the information, based on the query(S170_b). The electronic device 100 may provide the information invarious manners (S190_b). The electronic device 100 may provide theinformation in various manners to be described with reference to FIGS.22 through 29.

Hereinafter, detailed scenarios are described below.

[Scenario 1: Messaging Service]

FIGS. 3 through 7 illustrate detailed UI environments provided duringcommunication via a messaging service.

Referring to FIG. 3, the electronic device 100 may display communicationcontents between users via the messaging service.

The electronic device 100 may provide an interface including a backbutton 20, a voice call connection button 30, a user name box 40, and acommunication window (refer to reference numerals 5, 6, 7, and 8). Theuser name box 40 may include the back button 20 and the voice callconnection button 30.

A user may return to a previous menu by using the back button 20 (e.g.,touching or double-tapping the back button 20). The user may performvoice chatting by using the voice call connection button 30 (e.g.,touching or double-tapping the voice call connection button 30). Theuser name box 40 may display a name or a nickname of another party inconversation. Also, the user name box 40 may display a name or anickname of the user of the electronic device 100. The communicationwindow may display messages 5, 6, 7, and 8 that are exchanged betweenthe user and another party.

The electronic device 100 may determine whether to extract a keywordfrom each of messages 5, 6, 7, and 8, by using a natural languageprocessor 1200 (refer to FIG. 36). The electronic device 100 mayrecognize a meaning of each of the messages 5, 6, 7, and 8, and maydetermine whether to extract a keyword therefrom.

Also, the electronic device 100 may extract ‘pizza’ as a keyword via astatistical analysis and/or a semantic analysis. For example, theelectronic device 100 may extract ‘pizza’ as the keyword, based on 1)content of conversation between users, 2) a user's location log, 3)user's previously input information, and 4) history information about auser.

For example, the electronic device 100 may perform the natural languageanalysis on 1) the content of conversation between the users, and mayextract ‘pizza’ as the keyword. The electronic device 100 may recognizea meaning of a message included in the content of conversation betweenthe users, and may predict the user's need, based on the meaning of themessage. For example, the electronic device 100 may analyze the meaningof the message as ‘the user wants to eat pizza’, may predict, based onthe meaning, that 1) the user wants to find a pizza restaurant or 2) theuser needs a telephone number of the pizza restaurant near a currentlocation of the user, and may extract ‘pizza’ as the keyword.

Also, for example, the electronic device 100 may perform the naturallanguage analysis on the content of conversation, and may extract‘pizza’ as the keyword by further considering 2) the user's locationlog, 3) the user's previously input information, and 4) the historyinformation about the user.

For example, the user's location log may include a frequency of the userpassing a place where the user is currently located, the number of timesthat the user passed the place where the user is currently located, atotal time during which the user stays in the place where the user iscurrently located, a total time during which the user stays in the placewhere the user is currently located during a particular time period,and/or a time when the user first visits the current place.

For example, the user's previously input information may include anaddress of the user, a user's preference for a pizza restaurant, auser's tendency in selecting menu, and/or a place according to anestimated schedule of the user.

For example, the history information about the user may indicate ahistory about the user which is recorded in the electronic device 100and a server. Also, the history about the user may include a historyrecorded by the user when the user uses the electronic device 100, and ahistory recorded to the server, by the user, when the user uses an Appservice. Also, the history about the user may include content that isnot directly recorded by the user but is indirectly recorded when theuser uses the electronic device 100 or the APP service.

Referring to FIGS. 4A through 4C, the electronic device 100 maycontinuously monitor conversation content of the user, as shown in FIG.4, and may separately indicate or mark a keyword included in a message.

For example, the electronic device 100 may highlight the keywordincluded in a message as shown in FIG. 4A, may underline the keyword asshown in FIG. 4B, or may change a color of a speech balloon includingthe keyword as shown in FIG. 4C.

The electronic device 100 may allow the user to adjust, via settings, aduration time of the separate indication or mark on the keyword (referto FIG. 31E).

Although not illustrated, the electronic device 100 may indicate ormark, by using various methods, the keyword to differentiate the keywordfrom other contents of the message.

For example, the electronic device 100 may mark the keyword included inthe message, by using a different font, or putting a box around thekeyword.

For example, the electronic device 100 may mark the keyword included inthe message, by using a hyperlink.

For example, the electronic device 100 may mark the keyword included inthe message, by using a flickering effect.

For example, the electronic device 100 may mark the keyword included inthe message, by using an avatar.

For example, the electronic device 100 may mark the keyword included inthe message, by using a speech balloon.

For example, the electronic device 100 may mark the keyword included inthe message, by using a superscript or a subscript.

For example, the electronic device 100 may mark the keyword included inthe message, by using a color different from a color of other contentsof the message.

The above examples are given only for illustrative purposes and theexemplary embodiments are not limited thereto. The electronic device 100may indicate or mark the keyword by using any method to differentiatethe keyword from other contents of the message.

Referring to FIG. 5, the electronic device 100 may receive a user inputcorresponding to an indication or a mark that requires an input. Theuser may input a request by using various methods. The electronic device100 may receive an information provision request input.

The information provision request input may indicate an input by theuser to request the electronic device 100 to provide information. Forexample, the user may input the request by using various input methods(e.g., a touch, a click, a double click, etc.) to request the electronicdevice 100 to provide the information. Also, the user may input aparticular command including a voice and/or a text to request theelectronic device 100 to provide the information.

For example, the electronic device 100 may select the keyword byreceiving voice of the user via a microphone or a voice recognitionprocessor. For example, the electronic device 100 may select the keywordby receiving a click of the user via a mouse, a touch panel, or a touchpad. For example, the electronic device 100 may select the keyword byreceiving a touch of the user via the touch panel or the touch pad. Forexample, the electronic device 100 may select the keyword by receiving apreset particular gesture of the user via a sensing camera. For example,the electronic device 100 may receive an input of selecting the keywordvia a virtual keyboard or a virtual touchscreen.

Referring to FIGS. 6A through 6C, the electronic device 100 may providea query related to a keyword.

FIG. 6A illustrates a screen of the electronic device 100 that displaysa keyword (e.g., ‘pizza’) in a main box and displays candidate queriesin a candidate list window 610 below the main box. For example,candidate queries for the keyword ‘pizza’ may include ‘good day pizza’,‘good day pizza number’, ‘good day pizza location’, and ‘good day pizzamenu’.

The electronic device 100 may generate the candidate queries byperforming a statistical analysis and/or a semantic analysis, based onthe extracted keyword. As illustrated, the electronic device 100 maygenerate the candidate queries ‘good day pizza’, ‘good day pizzanumber’, ‘good day pizza location’, and ‘good day pizza menu’, basedon 1) content of conversation between users, 2) a user's location log,3) user's previously input information, and 4) history information abouta user.

The electronic device 100 may add a candidate query according to 1) thecontent of conversation between the users.

For example, the electronic device 100 may continuously perform naturallanguage processing on the content of conversation between the users andmay recognize a meaning of the content of conversation while theelectronic device 100 monitors the content of conversation between theusers. When the electronic device 100 recognizes that ‘pizza’ is a maintopic and the user wants to order ‘pizza’, according to the content ofconversation between the users, the electronic device 100 may predictthat the user needs a telephone number of a pizza restaurant. Also, if aword ‘good day pizza’ is used in the content of conversation between theusers, the electronic device 100 may add ‘good day pizza number’ as thecandidate query.

For example, when the electronic device 100 recognizes that ‘pizza’ isthe main topic and the user wants to order ‘good day pizza’, accordingto the content of conversation between the users, the electronic device100 may predict that the user needs information about a pizzarestaurant. Also, when a kind of pizza that the user wants to eat isincluded in the content of conversation between the users, theelectronic device 100 may add ‘good day pizza menu’ as the candidatequery.

The electronic device 100 may add a candidate query, according to 2) theuser's location log.

For example, the electronic device 100 recognizes that the user wants tofind a pizza restaurant near a current location or to have a telephonenumber of the pizza restaurant near the current location, and a recordthat the user paid a bill in the ‘good day pizza’ restaurant occursfrequent in payment detail reception messages in a history of the user,the electronic device 100 may add ‘good day pizza’ as the candidatequery.

For example, if the electronic device 100 recognizes that the user wantsto find a pizza restaurant near a current location, and a record thatthe user visited the ‘good day pizza’ restaurant occurs frequent in anaccumulated history of a user's location, the electronic device 100 mayadd ‘good day pizza location’ as the candidate query.

For example, if the electronic device 100 recognizes that the user wantsto have a telephone number of a pizza restaurant near a currentlocation, and a record that the user called the ‘good day pizza’restaurant occur frequent in a call history of the user, the electronicdevice 100 may add ‘good day pizza number’ as the candidate query.

The electronic device 100 may add a candidate query by analyzing 3) theuser's previously input information.

For example, the electronic device 100 may analyze the user's previouslyinput information, and may recognize that the ‘good day pizza’restaurant is a user's favorite pizza restaurant.

The user may directly record the ‘good day pizza’ restaurant as theuser's favorite pizza restaurant to the electronic device 100. Also, ifthe user recorded a plurality of pieces of information about the ‘goodday pizza’ restaurant to the electronic device 100, the electronicdevice 100 may recognize that the ‘good day pizza’ is the user'sfavorite pizza restaurant. If there is no direct record aboutinformation of the user's favorite pizza restaurant, the electronicdevice 100 may recognize, by performing the statistical analysis, thatthe ‘good day pizza’ restaurant is the user's favorite pizza restaurant,and may add ‘good day pizza number’ as the candidate query.

The electronic device 100 may determine, from information input to aserver via an application service, that the ‘good day pizza’ restaurantis the user's favorite pizza restaurant. The user may directly recordthe ‘good day pizza’ restaurant as the user's favorite pizza restaurantto a database of the server via the application service. Also, if theuser recorded a plurality of pieces of information about the ‘good daypizza’ restaurant to the database of the server via the applicationservice, the electronic device 100 may determine that the ‘good daypizza’ restaurant is the user's favorite pizza restaurant. If there isno direct record about information of the user's favorite restaurant inthe server, the electronic device 100 may recognize, by performing thestatistical analysis, that the ‘good day pizza’ restaurant is the user'sfavorite pizza restaurant, and may add ‘good day pizza number’ as thecandidate query.

The electronic device 100 may add a candidate query, according to 4) thehistory information about the user. The electronic device 100 may checka payment history of the user, and a plurality of records of paying abill in the ‘good day pizza’ restaurant are found, the electronic device100 may determine that the ‘good day pizza’ restaurant is the user'sfavorite pizza restaurant. When the electronic device 100 checks asearch history of the user, and detects that the user retrieved menus ofthe ‘good day pizza’ restaurant several times, the electronic device 100may add ‘good day pizza menu’ as the candidate query.

FIG. 6B illustrates a UI that displays, in an input box, a query that isan extracted keyword (e.g., ‘pizza’) as a default value. The keyword maybe determined by using various methods described with reference to FIGS.2A through 2C. For example, the electronic device 100 may generate thequery that is same as the determined keyword, and the query may beedited via an input interface provided by the electronic device 100.

FIG. 6C illustrates a UI that displays a query (e.g., ‘good day pizza’),as a default value, in an input box.

The query may be determined by using various methods described withreference to FIGS. 2A through 2C. For example, the query may begenerated based on a keyword and location information, as described withreference to FIG. 2C. For example, when the keyword is ‘pizza’, a word‘good day pizza’ is mentioned several times in messages between users,the electronic device 100 may generate ‘good day pizza’ as the query.

The keyword may be edited via an input interface provided by theelectronic device 100. For example, a user may edit the query ‘good daypizza’ to ‘good day pizza location’, ‘good day pizza number’, ‘good daypizza menu’, or the like. The electronic device 100 may obtaininformation, based on the query edited by the user.

Referring to FIG. 7, the electronic device 100 may provide a searchresult via screen conversion. Also, the electronic device 100 mayprovide a search result by using various methods to be described withreference to FIGS. 22 through 29. The electronic device 100 may provide,in a user name box, a messaging service button 4 for returning to themessaging service. Alternatively, the electronic device 100 may providea back button 2 for returning to the messaging service.

[Scenario 2: Voice Chatting Service]

FIGS. 8 through 10 illustrate detailed UI environments that are providedduring communication via a voice chatting service. Throughout thespecification, the voice chatting service may include, but not limitedto, a voice call service. Further, the voice chatting service mayinclude, but not limited to, a video call service.

Referring to FIG. 8, in an example 800 of the voice chatting service (orthe video call service), the electronic device 100 may displaycommunication contents between users via the voice chatting service, asillustrated in FIG. 3.

A user of the electronic device 100 may have a conversation with anotherparty via the voice chatting service. During the conversation betweenthe user and the other party, the user delivers a voice message ‘I wantto eat pizza’. The voice message of the user may be converted to a textby using a speech to text (STT) module (or STT engine), and may bedisplayed on a screen of the electronic device 100.

The electronic device 100 may determine whether to extract a keywordfrom each of messages which are converted to texts by using the STTmodule via the natural language processor 1200 (refer to FIG. 36). Theelectronic device 100 may recognize a meaning of each of the messages,and may determine whether to extract the keyword from each of themessages.

Also, the electronic device 100 may extract ‘pizza’ as the keyword byperforming a statistical analysis and/or a semantic analysis. A methodof extracting the keyword, the method being performed by the electronicdevice 100, may include the method described with reference to FIG. 3.

For example, the electronic device 100 may perform the natural languageanalysis on 1) content of conversation between users, and extract‘pizza’ as the keyword by further considering 2) a user's location log,3) user's previously input information, and 4) history information aboutthe user.

Similar to FIG. 4, the electronic device 100 may continuously monitorconversation content of the user, and may separately indicate or markthe keyword included in a message that is a keyword extraction target,in a similar manner to FIG. 4.

Also, similar to FIG. 5, the electronic device 100 may receive a userinput corresponding to an indication or a mark that requires an input.The user may input a request by using various methods as described withreference to FIG. 5.

Referring to FIGS. 9A and 9B, the electronic device 100 may provide aquery related to a keyword.

FIG. 9A illustrates an example 900 of the voice chatting service inwhich a UI that displays a keyword (e.g., ‘pizza’), as a default value,in an input box is provided. The keyword may be determined by usingvarious methods described in the operation S110 of FIG. 2A. Theelectronic device 100 may generate a query that is same as the keyword,and the query may be edited via an input interface provided by theelectronic device 100.

FIG. 9B illustrates an example 910 of the voice chatting service inwhich a UI that displays a query (e.g., ‘good day pizza’), as a defaultvalue, in an input box is provided. The query may be determined by usingvarious methods as described in the operation S130 of FIG. 2A. Thekeyword may be edited via an input interface provided by the electronicdevice 100. Also, the electronic device 100 may provide candidatequeries as shown in FIG. 6A, other than the methods shown in FIGS. 9Aand 9B.

Referring to FIG. 10, in an example 1010 of the voice chatting service,the electronic device 100 may partition a screen of the electronicdevice 100 into two areas 20 and 30, and may provide a search result tothe area 30 of the partitioned two areas 20 and 30. The electronicdevice 100 may partition the screen into the two areas 20 and 30, maydisplay a chatting App screen on the area 20 in an upper part of thescreen, and may display a search engine service screen on the area 30 ina lower part of the screen. Also, the user may select to display thechatting App screen on an entire portion of the screen by touching thechatting App screen in the upper part.

In addition, the electronic device 100 may provide the search result byusing various methods to be described with reference to FIGS. 22 through29.

[Scenario 3: Mailing Service]

FIGS. 11 through 15 illustrate detailed UI environments that areprovided during communication via a mailing service.

Referring to FIG. 11, the electronic device 100 may displaycommunication content between users via the mailing service.

A user of the electronic device 100 may communicate with the other partyvia e-mailing. An e-mail received by the user includes a comment aboutan appointment time and place.

The electronic device 100 may determine whether to extract a keywordfrom each of sentences in messages via the natural language processor1200 (refer to FIG. 36). The electronic device 100 may recognize ameaning of each of the messages, and may determine whether to extractthe keyword.

Also, the electronic device 100 may extract the keyword by performing astatistical analysis and/or a semantic analysis. A method of extractingthe keyword, the method being performed by the electronic device 100,may include the method described with reference to FIG. 3.

For example, the electronic device 100 may perform the natural languageanalysis on 1) content of conversation between users, and extract‘Shabumi’ as the keyword by further considering 2) a user's locationlog, 3) user's previously input information, and 4) history informationabout the user.

Similar to FIG. 4, the electronic device 100 may continuously monitorconversation content of the user, and may separately indicate or markthe keyword included in a message that is a keyword extraction target,in a similar manner to FIG. 4.

Referring to FIG. 12, similar to FIG. 5, the electronic device 100 mayreceive a user input corresponding to an indication or a mark. The usermay input a request by using various methods as described with referenceto FIG. 5.

Referring to FIG. 13, the electronic device 100 may provide a list ofApps related to the keyword. For example, the electronic device 100 maycheck a category of the keyword, and if the keyword is a word related toa place, the electronic device 100 may recognize the place as thecategory, and may provide a list of Apps related to a map.

In another exemplary embodiment, the electronic device 100 may notprovide the list of Apps related to the keyword, and may obtaininformation by using a preset App.

FIG. 14 illustrates a UI that displays a keyword (e.g., ‘Shabumi’), as adefault value, in an input box. The keyword may be determined by usingvarious methods described in the operation S110 of FIG. 2A. For example,the electronic device 100 may generate a query that is same as thekeyword. The query may be determined by using various methods asdescribed in the operation S130 of FIG. 2A.

The query may be edited via an input interface. For example, theelectronic device 100 may extract ‘Shabumi’ as the keyword, and maygenerate the query as ‘Shabumi’ same as the keyword. A user may edit thequery to ‘Shabumi Dogok branch’ via the input interface. Also, theelectronic device 100 may automatically generate the query as ‘Shabumirestaurant’. The electronic device 100 may obtain information about alocation of ‘Shabumi restaurant’ via an App service.

Referring to FIG. 15, the electronic device 100 may provide a searchresult via screen partition. Also, the electronic device 100 may providethe search result by using various methods to be described withreference to FIGS. 22 through 29.

[Scenario 4: Social Network Service]

FIGS. 16 through 19 illustrate detailed UI environments that areprovided during communication via an SNS.

Referring to FIG. 16, the electronic device 100 may displaycommunication content between users via the SNS.

The electronic device 100 may display contents of a memo of anotherparty via the SNS. As shown in FIG. 16, from the contents ‘watchingbatman with Sumin in L.A.’ of the memo of the other party (Jung), wordssuch as ‘batman’, ‘Sumin’, and ‘L.A.’ may be extracted as keywords. Theelectronic device 100 may determine whether to extract a keyword fromeach of sentences included in messages via the natural languageprocessor 1200 (refer to FIG. 36). The electronic device 100 mayrecognize a meaning of each of the messages, and may determine whetherto extract the keyword.

Also, the electronic device 100 may extract ‘batman’, ‘Sumin’, and‘L.A.’ as the keywords by performing a statistical analysis and/or asemantic analysis. A method of extracting the keyword, the method beingperformed by the electronic device 100, may include the method describedwith reference to FIG. 3.

For example, the electronic device 100 may perform the natural languageanalysis on 1) content of conversation between users, and extract‘batman’, ‘Sumin’, and ‘L.A.’ as the keywords by further considering 2)a user's location log, 3) user's previously input information, and 4)history information about the user.

Also, similar to FIG. 4, the electronic device 100 may continuouslymonitor conversation content of the user, and may separately indicate ormark the keyword included in a message that is a keyword extractiontarget, in a similar manner to FIG. 4.

Referring to FIG. 17, similar to FIG. 5, the electronic device 100 mayreceive a user input corresponding to an indication or a mark. The usermay input a request by using various methods as described with referenceto FIG. 5. The user may select a query from among the keywords. When theuser selects ‘batman’ as the keyword, the electronic device 100 mayobtain information about ‘batman’ via an App service.

Referring to FIG. 18, the electronic device 100 may provide a list ofApps related to the keyword. For example, the electronic device 100 maycheck a category of the keyword, and if the keyword is a word related toa movie, the electronic device 100 may recognize the movie as thecategory, and may provide a list of Apps related to the movie.

Referring to FIG. 19, the electronic device 100 may provide a searchresult via screen conversion. Also, the electronic device 100 mayprovide the search result by using various methods to be described withreference to FIGS. 22 through 29.

[Scenario 5: Scheduler]

FIG. 20 illustrates a detailed UI environment that is provided while ascheduler service is used.

Referring to FIG. 20, the electronic device 100 may be a tablet PC. Theelectronic device 100 may provide a scheduler App service to a user. Theuser may manage his/her time schedules by using the scheduler Appservice.

For example, it is assumed that the user has a schedule registered withthe scheduler APP for Sep. 24, 2013. Also, it is assumed that theregistered schedule indicates that the user tours Kyung Bok Palace with‘Kaur’ and has dinner with ‘Kaur’.

The electronic device 100 may recognize, from a call history of theuser, that the user did not call restaurants near Kyung Bok Palace, andmay recognize, from an internet search history of the user, that theuser did not search the restaurants near Kyung Bok Palace. Also, theelectronic device 100 may recognize, by using a GPS module included inthe electronic device 100, that the user did not visit Kyung Bok Palace.In this case, the electronic device 100 may extract, as a keyword,‘Kyung Bok’ recorded on the scheduler App service, or ‘

’ that is a Korean character for Kyung Bok Palace, may automaticallygenerate ‘tasty restaurant near Kyung Bok Palace’ as a query, and mayprovide an indication or a mark on ‘Kyung Bok’ recorded on the schedulerApp service.

When the user selects a marked (e.g., highlighted) text ‘Kyung Bok’, theelectronic device 100 may provide the generated query ‘tasty restaurantnear Kyung Bok Palace’ to a search service, may obtain information aboutthe query in real-time, and may provide the obtained information. Also,the electronic device 100 may retrieve information about the generatedquery ‘tasty restaurant near Kyung Bok Palace’, may store the obtainedinformation in the electronic device 100, and may provide the storedinformation to the user whenever the user wants.

[Scenario 6: Word Processor]

FIG. 21 illustrates a detailed UI environment that is provided while aword processor App is used.

Referring to FIG. 21, the electronic device 100 may be a laptopcomputer. The electronic device 100 may provide a word processor Appservice to a user. The user may generate and edit a document by usingthe word processor App.

For example, the user may edit a news article that is written inEnglish. The electronic device 100 may recognize an English vocabularylevel of the user, in consideration of user information, and mayindicate or mark a word that is likely difficult for the user tounderstand.

FIGS. 22A through 27B illustrate methods of providing obtainedinformation to a user via an output interface, the methods beingperformed by the electronic device 100.

The electronic device 100 may provide the obtained information to theuser via the output interface in various manners as described withreference to Table 4.

FIGS. 22A through 22I illustrate examples in which, when the electronicdevice 100 is a smartphone or a tablet PC and a user performs textchatting by using the smartphone or the tablet PC, information obtainedby the electronic device 100 is provided to the user via an outputinterface.

FIG. 22A illustrates a situation in which users perform text chatting byusing electronic devices 100 and 105.

Referring to FIG. 22A, a first user of the electronic device 100 and asecond user of the electronic device 105 have a conversation via thetext chatting. During the conversation between the first user and thesecond user, the first user delivers a message ‘I want to eat pizza’ tothe second user.

Referring to a chatting message 7, Chris who is the second user of theelectronic device 105 asks a question ‘What do you want to do?’ toHyunjin who is the first user of the electronic device 100 at 8:28 A.M.,Aug. 6, 2013. Referring to a chatting message 8, Hyunjin who is thefirst user of the electronic device 100 replies to Chris who is thesecond user of the electronic device 105 that ‘I want to eat pizza’ at8:30 A.M., Aug. 6, 2013. Thus, at time 8:30 A.M., Aug. 6, 2013, theconversation is ongoing.

While the electronic device 100 monitors the conversation between thefirst and second users, the electronic device 100 may extract a keywordfrom a message for which additional information is needed, and mayprovide a query to the first user. The first user may edit the providedquery. The electronic device 100 may obtain information, based on theedited query and location information of the electronic device 100. Theelectronic device 100 may provide the information to the first user viathe output interface in manners shown in FIGS. 22B through 22I.

FIG. 22B illustrates an example in which information obtained by theelectronic device 100 of FIG. 22A is provided to the first user viascreen conversion.

As illustrated in FIG. 22B, the electronic device 100 may convert a textchatting interface screen, which is displayed on a screen of theelectronic device 100, to a screen that displays the obtainedinformation. For example, the electronic device 100 may close the textchatting interface screen displayed on the screen, and may display theobtained information and an interface of a search engine providing theobtained information. The first user may return to the text chattinginterface screen. The first user may return to the text chattinginterface screen by touching or double tapping a user name box 4 at atop of the screen.

FIG. 22C illustrates an example in which information obtained by theelectronic device 100 of FIG. 22A is provided to the first user viascreen partition.

As illustrated in FIG. 22C, the electronic device 100 may convert aportion of an image of a text chatting interface, which is displayed ona screen of the electronic device 100, to an image that displays theobtained information. For example, the electronic device 100 may close aportion of a text chatting interface screen displayed on the screen ofthe electronic device 100, and may display the obtained information andan interface of a search engine providing the obtained information. Thefirst user may return to the text chatting interface screen by selectinga back button 2 at a top of the screen of the electronic device 100.

FIG. 22D illustrates an example in which information obtained by theelectronic device 100 of FIG. 22A is provided to the first user by usingvoice.

As illustrated in FIG. 22D, the electronic device 100 may continuouslydisplay an image of a text chatting interface which is displayed on ascreen of the electronic device 100, and may provide the obtainedinformation by using voice. For example, the electronic device 100 maycontinuously display the image of the text chatting interface which isdisplayed on the screen of the electronic device 100, may summarize theobtained information, and may provide the information by outputtingvoice corresponding to speech data ‘a result obtained by searching for aquery ‘good day pizza’ is A, B, and C’, via a speaker.

A type of a language provided by the electronic device 100 may vary,including Korean, English, French, German, Spanish, Japanese, Chinese,etc. For example, a type of a language provided by the electronic device100 may be set by the first user via settings. For example, types of alanguage provided by the electronic device 100 may be provided by theelectronic device 100.

FIG. 22E illustrates an example in which information obtained by theelectronic device 100 of FIG. 22A is provided to the first user viavoice and screen partition.

As illustrated in FIG. 22E, the electronic device 100 may convert aportion of an image of a text chatting interface, which is displayed ona screen of the electronic device 100, to an image that displays theobtained information, and may provide the obtained information by usingvoice. Also, for example, the electronic device 100 may convert theportion of the image of the text chatting interface, which is displayedon the screen of the electronic device 100, to the image that displaysthe obtained information, and may display to the first user a messageinquiring whether to summarize and provide a portion of the obtainedinformation by using voice. For example, the electronic device 100 maydisplay to the first user a question “Do you want to see an image resultobtained by searching for a query ‘good day pizza’?”.

FIG. 22F illustrates an example in which information obtained by theelectronic device 100 of FIG. 22A is provided to one or more users via amessaging service.

As illustrated in FIG. 22F, the electronic device 100 may continuouslydisplay an image of a text chatting interface on a screen of theelectronic device 100, and a messaging service server 2200 may providethe obtained information to the electronic device 100 of the first userand/or the electronic device 105 of the second user via the messagingservice. Also, for example, the electronic device 100 may continuouslydisplay the image of the text chatting interface on the screen of theelectronic device 100 to allow the first user and the second usercontinue to perform text chatting, and the messaging service server 2200may transmit the obtained information to a particular telephone numberof one or more of a third user via the messaging service. For example,the messaging service server 2200 may retrieve the particular telephonenumber from a messaging database 2201.

FIG. 22G illustrates an example in which information obtained by theelectronic device 100 of FIG. 22A is provided to one or more users viaan e-mail service.

As illustrated in FIG. 22G, the electronic device 100 may continuouslydisplay an image of a text chatting interface on a screen of theelectronic device 100, and an e-mail server 2210 may provide theobtained information to an e-mail address of the first user and/or ane-mail address of the second user via the e-mail service. Also, forexample, the electronic device 100 may continuously display the image ofthe text chatting interface on the screen of the electronic device 100to allow the first user and the second user to continue to perform textchatting, and the e-mail server 2210 may transmit the obtainedinformation to a particular e-mail account of one or more of a thirduser via the e-mail service. For example, the e-mail server 2210 mayretrieve the particular e-mail account from an e-mail database 2211.

FIG. 22H illustrates an example in which information obtained by theelectronic device 100 of FIG. 22A is provided to one or more users viaan SNS.

As illustrated in FIG. 22G, the electronic device 100 may continuouslydisplay an image of a text chatting interface on a screen of theelectronic device 100, and an SNS server 2220 may provide the obtainedinformation to an SNS account of the first user and/or an SNS account ofthe second user via the SNS. Also, for example, the electronic device100 may continuously display the image of the text chatting interface onthe screen of the electronic device 100 to allow the first user and thesecond user to continue to perform text chatting, and the SNS server2220 may transmit the obtained information to a message board of aparticular SNS account of one or more of a third user via the e-mailservice. For example, the SNS server 2220 may retrieve the particularSNS account from an e-mail database 2221.

FIG. 22I illustrates an example in which information obtained by theelectronic device 100 of FIG. 22A is provided to one or more users viaan App service.

As illustrated in FIG. 22I, the electronic device 100 may continuouslydisplay an image of a text chatting interface on a screen of theelectronic device 100, and an application service server 2230 mayprovide the obtained information to an App account of the first userand/or an App account of the second user via the App service. Also, forexample, the electronic device 100 may continuously display the image ofthe text chatting interface on the screen of the electronic device 100to allow the first user and the second user to continue to perform textchatting, and the application service server 2230 may transmit theobtained information to a message board of a particular App account ofone or more of a third user via the App service. For example, theapplication service server 2230 may retrieve the particular App accountfrom a database 2231.

FIGS. 23A through 23E illustrate examples in which, when the electronicdevice 100 is a smartphone or a tablet PC and a user performs voicechatting by using the smartphone or the tablet PC, information obtainedby the electronic device 100 is provided to the user via an outputinterface.

FIG. 23A illustrates a situation in which users perform voice chattingby using electronic devices 100 and 105.

Referring to FIG. 23A, a first user of the electronic device 100 and asecond user of the electronic device 105 have a conversation via thevoice chatting. During the conversation between the first user and thesecond user, the first user delivers a voice message ‘I want to eatpizza’. The voice message of the first user may be converted to a textby using an STT module, and may be displayed on a screen of theelectronic device 100.

While the electronic device 100 monitors the conversation, theelectronic device 100 may extract a keyword from a message for whichadditional information is needed and may provide a query to the firstuser. The first user may edit the provided query. The electronic device100 may obtain information, based on the edited query and locationinformation of the electronic device 100. The electronic device 100 mayprovide the information to the first user via the output interface inmanners shown in FIGS. 23B through 23E.

FIG. 23B illustrates an example in which information obtained by theelectronic device 100 of FIG. 23A is provided to the first user viascreen conversion.

As illustrated in FIG. 23B, the electronic device 100 may convert avoice chatting interface screen, which is displayed on a screen of theelectronic device 100, to a screen that displays the obtainedinformation. For example, the electronic device 100 may close the voicechatting interface screen displayed on the screen, and may display theobtained information and an interface of a search engine providing theobtained information.

FIG. 23C illustrates an example in which information obtained by theelectronic device 100 of FIG. 23A is provided to the first user viascreen partition.

As illustrated in FIG. 23C, the electronic device 100 may convert aportion of a screen of a voice chatting interface, which is displayed ona screen of the electronic device 100, to a screen that displays theobtained information. For example, the electronic device 100 may close aportion of a voice chatting interface screen displayed on the screen ofthe electronic device 100, and may display the obtained information andan interface of a search engine providing the obtained information.

FIG. 23D illustrates an example in which information obtained by theelectronic device 100 of FIG. 23A is provided to the first user by usingvoice.

As illustrated in FIG. 23D, the electronic device 100 may continuouslydisplay a screen of a voice chatting interface on a screen of theelectronic device 100, and may provide the obtained information by usingvoice. For example, the electronic device 100 may continuously displaythe screen of the voice chatting interface on the screen of theelectronic device 100, may summarize the obtained information, and mayprovide the information by using voice.

FIG. 23E illustrates an example in which information obtained by theelectronic device 100 of FIG. 23A is provided to the first user viavoice and screen partition.

As illustrated in FIG. 23E, the electronic device 100 may convert aportion of a screen of a voice chatting interface, which is displayed ona screen of the electronic device 100, to a screen that displays theobtained information, and may provide the obtained information by usingvoice. Also, for example, the electronic device 100 may convert theportion of the screen of the voice chatting interface, which isdisplayed on the screen of the electronic device 100, to the screen thatdisplays the obtained information, may summarize a portion of theobtained information, and may provide the information by using voice.

Also, during the voice chatting, the obtained information may betransmitted to the first user, the second user, and/or to one or more ofa third user, via a messaging service, an e-mail service, an SNS, and/oran App service, similar to the exemplary embodiments described abovewith respect to the text chatting between the users.

Here, the obtained information may be transmitted to at least user ofthe first user, the second user, and/or to one or more of a third userdepending on whether the at least one user uses the messaging service,the at least one user uses the e-mail service, the at least one useruses the SNS, and/or the at least one user uses the App service.

FIGS. 24A through 24F illustrate examples in which, when the electronicdevice 100 is a smartwatch and a user performs text chatting by usingthe smartwatch, information obtained by the electronic device 100 isprovided to the user via an output interface.

FIG. 24A illustrates a situation in which users perform text chatting byusing smartwatches that are electronic devices 100 and 105.

Referring to FIG. 24A, a first user of the electronic device 100 and asecond user of the electronic device 105 have a conversation via thetext chatting. During the conversation between the first user and thesecond user, the first user delivers a message ‘I want to eat pizza’.While the electronic device 100 monitors the conversation, theelectronic device 100 may extract a keyword from a message for whichadditional information is needed, and may provide a query to the firstuser. The first user may edit the provided query. The electronic device100 may obtain information, based on the edited query and locationinformation of the electronic device 100. According to exemplaryembodiments, the electronic device 100 may provide the information tothe first user via the output interface in manners as shown in FIGS. 24Bthrough 24F.

FIG. 24B illustrates an example in which primary information from amonga plurality of pieces of information obtained by the electronic device100 of FIG. 24A is provided to the first user.

As illustrated in FIG. 24B, the electronic device 100 may convert a textchatting interface screen, which is displayed on a screen of theelectronic device 100, to a screen that displays the primary informationfrom among the plurality of pieces of obtained information. For example,the electronic device 100 may display one of a plurality of pieces ofprimary information (e.g., images shown in FIG. 24C) from among theplurality of pieces of obtained information.

FIG. 24C illustrates an example of a method of providing a user withprimary information from among a plurality of pieces of informationobtained by the electronic device 100 of FIG. 24A, the method beingperformed by the electronic device 100.

As illustrated in FIG. 24C, the electronic device 100 may display one ofa plurality of pieces of primary information (e.g., the images as shownin (a), (b), (c), (d), and (e) of FIG. 24C) from among the plurality ofpieces of obtained information. For example, the first user may checkeach of the plurality of pieces of primary information by using a shiftpage button 21 provided by the electronic device 100.

FIG. 24D illustrates an example in which information obtained by theelectronic device 100 of FIG. 22A is provided to the first user by usingvoice.

As illustrated in FIG. 24D, the electronic device 100 may continuouslydisplay an image of a text chatting interface which is displayed on ascreen of the electronic device 100, and may provide the obtainedinformation by using voice. For example, the electronic device 100 maycontinuously display the image of the text chatting interface which isdisplayed on the screen of the electronic device 100, may summarize theobtained information, and may provide the information by using voice.

FIG. 24E illustrates an example in which primary image information fromamong image information obtained by the electronic device 100 of FIG.22A is provided to the first user by using voice.

As illustrated in FIG. 24E, the electronic device 100 may convert a textchatting interface screen, which is displayed on a screen of theelectronic device 100, to a screen that displays the primary imageinformation from among the obtained image information, and may providethe obtained image information by using voice. Also, for example, theelectronic device 100 may convert a portion of the text chattinginterface screen, which is displayed on the screen of the electronicdevice 100, to a screen that displays the obtained image information,may summarize a portion of the obtained image information, and mayprovide the image information by using voice.

FIG. 24F illustrates an example in which primary text information fromamong text information obtained by the electronic device 100 of FIG. 22Ais provided to the first user by using voice.

As illustrated in FIG. 24F, the electronic device 100 may convert a textchatting interface screen, which is displayed on a screen of theelectronic device 100, to a screen that displays the primary textinformation from among the obtained text information, and may providethe obtained text information by using voice. Also, for example, theelectronic device 100 may convert a portion of the text chattinginterface screen, which is displayed on the screen of the electronicdevice 100, to a screen that displays the obtained text information, maysummarize a portion of the obtained text information, and may providethe text information by using voice.

FIGS. 25A and 25B illustrate examples in which, when the electronicdevice 100 is a smartwatch, and a user performs text chatting by usingthe smartwatch 100 and a smartphone that is an electronic device 101interoperating with the smartwatch 100, information obtained by theelectronic device 100 is provided to the user via an output interface.

Referring to FIG. 25A, the smartwatch 100 and the smartphone 101 mayinteroperate with each other. For example, the smartwatch 100 may beconnected with the smartphone 101 via short-distance communication. Forexample, the smartwatch 100 and the smartphone 101 may share a portionof a display screen. For example, when the smartphone 101 receives amessage from a server (not shown), the smartwatch 100 may notify a userin a vibration manner that the message is received. For example, whenthe smartphone 101 receives a voice call from the server, the smartwatch100 may notify the user in a vibration manner that the voice call isreceived.

For example, some of messages received by the electronic device 101 maybe displayed on the electronic device 100. Also, some of messages thatare continuously exchanged between the electronic device 100 and anelectronic device 105 may be displayed on the electronic device 101 andan electronic device 106, respectively. For example, the electronicdevice 106 may be a smartphone that interoperates with the electronicdevice 105 that may be a smartwatch. A first user of the electronicdevice 101 may check, via a touchscreen, the messages exchanged betweenthe first user and a second user of the electronic device 105 and theelectronic device 106. For example, the first user of the electronicdevice 101 may vertically scroll the messages exchanged between thefirst and second users, by performing a touch input such as panning orflick on the touchscreen.

The first user of the electronic device 100 and the electronic device101, and the second user of the electronic device 105 and the electronicdevice 106 have a conversation via the text chatting. During theconversation between the first user and the second user, the first userdelivers a message ‘I want to eat pizza’. In this situation, theelectronic device 100 may display a relatively small portion of theconversation, and the electronic device 101 may display a relativelylarge portion of the conversation.

While the electronic device 100 monitors the conversation, theelectronic device 100 may extract a keyword from a message for whichadditional information is needed, and may provide a query to the firstuser. The first user may edit the provided query. The electronic device100 may obtain information, based on the edited query and locationinformation of the electronic device 100.

The electronic device 100 may continuously analyze the conversationbetween the first user and the second user, and may monitor each ofmessages. Also, while the electronic device 100 monitors each of themessages, the electronic device 100 may determine whether each of themessages is an inquiring message, and may determine whether it is neededto obtain information for each of the messages. Also, the electronicdevice 100 may extract a keyword from each of the messages. Also, theelectronic device 100 may continuously and differently mark a part ofeach message from other parts of each message, wherein the partcorresponds to the extracted keyword. Also, the electronic device 100may maintain the marking of each message for a preset time period. Also,the electronic device 100 may adjust, via settings, the marking of eachmessage to be maintained for the preset time period and then to beunmarked after the preset time period (refer to FIG. 30E).

Also, the electronic device 100 may display the marking of each messagevia an output interface to allow the first user check the marking via,for example, a scroll. Also, when the first user performs a touch inputon the marking of each message, the electronic device 100 may obtaininformation about the marking of each message, and may display theinformation via the output interface. Also, when the first user performsa touch input on the marking of each message, the electronic device 100may display information via the output interface which is previouslyobtained with respect to the marking of each message.

The electronic device 100 may provide the information to the first userin a manner as shown in FIG. 25B via the output interface.

FIG. 25B illustrates an example in which information obtained by theelectronic device 100 of FIG. 25A is provided to the first user via theelectronic device 100 and the electronic device 101 that interoperateswith the electronic device 100.

As illustrated in FIG. 25B, the electronic device 100 may convert a textchatting interface screen, which is displayed on a screen of theelectronic device 100, to a screen that displays primary informationfrom among the obtained information. Also, the electronic device 101 mayconvert the text chatting interface screen, which is displayed on ascreen of the electronic device 101, to a screen that displays theobtained information. For example, the electronic device 100 may displayone of a plurality of pieces of primary information (e.g. each of imagesas shown in (a)-(e) as shown in FIG. 24C) from among the obtainedinformation. Further, the electronic device 101 may close the textchatting interface screen displayed on the screen, and may display theobtained information and an interface of a search engine that providedthe obtained information.

FIGS. 26A and 26B illustrate examples in which, when the electronicdevice 100 is a smartwatch 100 and a user performs voice chatting byusing the smartwatch 100, information obtained by the electronic device100 is provided to the user via an output interface.

Referring to FIG. 26A, a first user of the electronic device 100 and theelectronic device 101, which interoperates with the electronic device100, and a second user of the electronic device 105 and the electronicdevice 106, which interoperates with the electronic device 105, have aconversation via the voice chatting. During the conversation between thefirst user and the second user, the first user delivers a message ‘Iwant to eat pizza’. In this situation, the electronic device 100 maydisplay a call image, and the electronic device 101 may convert a voicemessage to a text message via STT conversion and may display the textmessage.

While the electronic device 100 monitors the conversation, theelectronic device 100 may extract a keyword from a message for whichadditional information is needed, and may provide a query to the firstuser. The first user may edit the provided query. The electronic device100 may obtain information, based on the edited query and locationinformation of the electronic device 100. The electronic device 100 mayprovide the information to the first user in a manner as shown in FIG.26B via the output interface.

FIG. 26B illustrates an example in which information obtained by theelectronic device 100 of FIG. 26A is provided to the first user via theelectronic device 100 and the electronic device 101 that interoperateswith the electronic device 100.

As illustrated in FIG. 26B, the electronic device 100 may convert avoice chatting interface screen 2600, which is displayed on a screen ofthe electronic device 100, to a screen 2601 that displays primaryinformation from among the obtained information. Also, the electronicdevice 101 may convert a text chatting interface screen, which isdisplayed on a screen of the electronic device 101, to a screen 2603that displays the obtained information. For example, the electronicdevice 100 may display one of a plurality of pieces of primaryinformation (e.g. each of the images (a)-(e) shown in FIG. 24C) fromamong the obtained information. Further, the electronic device 101 mayclose the text chatting interface screen displayed on the screen, andmay display the obtained information and an interface of a search enginethat provided the obtained information.

FIGS. 27A and 27B illustrate examples in which, when the electronicdevice 100 is smartglasses and a user performs text chatting by usingthe smartglasses, information obtained by the electronic device 100 isprovided to the user via an output interface.

Referring to FIG. 27A, a first user of the electronic device 100 and asecond user of the electronic device 105 have a conversation via thetext chatting. During the conversation between the first user and thesecond user, the first user delivers a message ‘I want to eat pizza’.

The first user and the second user may perform the text chatting byusing an input interface of the electronic device 100 and an inputinterface of the electronic device 105, respectively. For example, theinput interface may include a virtual keyboard, a virtual touchscreen,or the like.

Also, the first user and the second user may perform the text chattingby using an output interface of the electronic device 100 and an outputinterface of the electronic device 105, respectively. For example, theoutput interface may include a mini-projector and prism, or the like.

While the electronic device 100 monitors the conversation, theelectronic device 100 may extract a keyword from a message for whichadditional information is needed, and may provide a query to the firstuser. The first user may edit the provided query. The electronic device100 may obtain information, based on the edited query and locationinformation of the electronic device 100. The electronic device 100 mayprovide the information to the first user in a manner as shown in FIG.27B via the output interface

FIG. 27B illustrates an example in which information obtained by theelectronic device 100 of FIG. 27A is provided to the first user via theelectronic device 100.

As illustrated in FIG. 27B, the electronic device 100 may convert a textchatting interface screen, which is displayed on a screen of theelectronic device 100, to a screen that displays the obtainedinformation.

FIGS. 28A and 28B illustrate examples in which, when the electronicdevice 100 is smartglasses and a user performs voice chatting by usingthe smartglasses, information obtained by the electronic device 100 isprovided to the user via an output interface.

Referring to FIG. 28A, a first user of the electronic device 100 and asecond user of the electronic device 105 have a conversation via thevoice chatting. During the conversation between the first user and thesecond user, the first user delivers a message ‘I want to eat pizza’.

Also, the first user and the second user may perform the voice chattingby using an output interface of the electronic device 100 and an outputinterface of the electronic device 105, respectively. For example, theoutput interface may include a mini-projector and prism, or the like.The electronic device 100 may provide the information to the first userin a manner as shown in FIG. 28B via the output interface.

FIG. 28B illustrates an example in which information obtained by theelectronic device 100 of FIG. 28A is provided to the first user via theelectronic device 100.

As illustrated in FIG. 28B, the electronic device 100 may convert avoice chatting interface screen, which is displayed on a screen of theelectronic device 100, to a screen that displays the obtainedinformation.

FIGS. 29A and 29B illustrate examples in which, when the electronicdevice 100 includes various electronic devices including smartglasses,and a user communicates with another user via at least one of voicechatting and text chatting by using the electronic devices, informationobtained by the electronic device 100 is provided to the user via anoutput interface.

As illustrated in FIG. 29A, a plurality of users may communicate viavarious types of chatting described above with reference to FIGS. 22through 28, and as illustrated in FIG. 29B, the plurality of users mayreceive information obtained by the electronic device 100 via varioustypes of chatting described above with reference to FIGS. 22 through 28.

For example, a first user may perform text chatting by using asmartwatch in a manner described with reference to FIG. 25, and a seconduser may perform text chatting by using smartglasses in a mannerdescribed with reference to FIG. 27.

For example, the first user may perform the text chatting by using thesmartwatch in a manner described with reference to FIG. 25, and thesecond user may perform voice chatting by using smartglasses in a mannerdescribed with reference to FIG. 28. In this case, a voice message inputby the second user may be converted to a text message via STT conversionand may be delivered to the first user. Also, in another exemplaryembodiment, a text message input by the first user may be converted to avoice message and may be delivered to the second user.

For example, the chatting in the aforementioned combined manners may beused in communication such as a video conference among at least threeusers.

Also, for example, the chatting in the aforementioned combined mannersmay be applied to various communication services such as an SNS whichare provided via applications, and various communication services suchas an e-mail service which are provided via a server.

The electronic device 100 may provide information to a user in variousmanners described with reference to FIGS. 22 through 29.

FIGS. 30A through 31B illustrate settings UIs that are provided by theelectronic device 100, according exemplary embodiments.

Referring to FIG. 30A, the electronic device 100 may provide a user witha settings UI with respect to ON/OFF of an information providingoperation. The user may select, via the settings UI, whether to receiveinformation according to the information providing methods according tothe exemplary embodiments. The user may set ON/OFF of the informationproviding operation according to the exemplary embodiments.

Setting of the electronic device 100 may include presenting operationsof the electronic device 100 to receive a setting input by the user, toprovide a use environment of the electronic device 100 based on thesetting input.

While FIG. 30A illustrates presenting an operation of query generation,the query generation operation may be referred to as different namessuch as, for example, an autonomic search engine, an autonomicinformation search, or the like, according to applied products, and thusthe name of the operation does not limit the scope of the inventiveconcept.

The electronic device 100 may provide a further detailed settings UI tothe user. For example, when the information providing operation is setas ON, the electronic device 100 may allow the user to select a methodof obtaining information.

For example, the electronic device 100 may allow the user to selectwhether to obtain information by performing an App search or an internetsearch. Also, the electronic device 100 may obtain information bysimultaneously performing the App search and the internet search.

Referring to FIG. 30B, while the electronic device 100 provides the userwith the settings UI with respect to ON/OFF of the information providingoperation, the electronic device 100 may differently provide differentlydetailed settings UI to the user.

For example, when the information providing operation is set as ON, theelectronic device 100 may allow the user to select how information isprovided. For example, the electronic device 100 may allow the user toselect whether to provide information in a text, voice, or an image.

For example, when the information providing operation is set as ON, theelectronic device 100 may allow the user to select how edited query isreceived. For example, the electronic device 100 may allow the user toselect whether to receive the edited query in a text or voice.

Referring to FIG. 30C, while the electronic device 100 provides the userwith the settings UI with respect to ON/OFF of the information providingoperation, the electronic device 100 may differently provide detailedsettings UI to the user.

For example, when the information providing operation is set as ON, theelectronic device 100 may allow the user to select to whose messageinformation is to be provided. For example, the electronic device 100may be set to extract a keyword only from a message of a first user User1 and to provide information about the keyword.

Alternatively, for example, when the information providing operation isset as ON, the electronic device 100 may allow the user to select towhom information is to be provided. For example, the electronic device100 may be set to provide information to the first user User 1 and asecond user User 2.

Referring to FIG. 30D, while the electronic device 100 provides the userwith the settings UI with respect to ON/OFF of the information providingoperation, the electronic device 100 may differently provide detailedsettings UI to the user.

For example, when the information providing operation is set as ON, theelectronic device 100 may allow the user to select which information isreferred to in extracting a keyword. Alternatively, for example, whenthe information providing operation is set as ON, the electronic device100 may allow the user to select which information is referred to ingenerating a query. Alternatively, for example, when the informationproviding operation is set as ON, the electronic device 100 may allowthe user to select which information is referred to in obtaininginformation based on an edited query and location information of theelectronic device 100.

Referring to FIG. 30E, while the electronic device 100 provides the userwith the settings UI with respect to ON/OFF of the information providingoperation, the electronic device 100 may differently provide detailedsettings UI to the user.

For example, when the information providing operation is set as ON, theelectronic device 100 may allow the user to select a keyword displayingduration time. For example, the user may set the keyword displayingduration time as one minute, and then, after one minute elapses when ahighlighted keyword is displayed, the electronic device 100 may stophighlighting the keyword.

Referring to FIG. 30F, when the electronic device 100 obtainsinformation based on an edited query and location information of theelectronic device 100, the electronic device 100 may allow the user toselect a search engine to be used. For example, when the electronicdevice 100 obtains the information based on the edited query and thelocation information of the electronic device 100, the electronic device100 may be set to select a commercialized internet search engine such asGoogle, Bing, Yahoo, or Naver.

Referring to FIG. 30G, the electronic device 100 may provide the userwith a settings UI with respect to setting a field of interest. Also,while the electronic device 100 provides the user with the settings UIwith respect to setting the field of interest, the electronic device 100may differently provide detailed settings UI to the user.

For example, the electronic device 100 may allow the user to select acategory of a keyword when the keyword is extracted. Alternatively, forexample, when the information providing operation is set as ON, theelectronic device 100, the electronic device 100 may allow the user toselect a category of information when the information is obtained basedon an edited query and location information of the electronic device100.

Referring to FIG. 31A, during communication, the electronic device 100may provide a user with a settings switch 10 with respect to ON/OFF ofan information providing operation by using an interface of thecommunication. The user may determine whether to activate theinformation providing operation, by using the settings switch 10. Theuser may set the settings switch 10 as ON, and thus may set theelectronic device 100 to perform the information providing operationaccording to the inventive concept. The user may set the settings switch10 as OFF, and thus may set the electronic device 100 not to perform theinformation providing operation according to the inventive concept.

Referring to FIG. 31B, during communication, the electronic device 100may provide a user with a settings bar 10_a with respect to a user input(a text, an image, and voice) by using an interface of thecommunication. The user may select, by using the settings bar 10_a, howto receive the user input.

For example, the user may set the settings bar 10_a as ‘Text’, and thusmay set the electronic device 100 to search for a query via texts. Also,the user may set the settings bar 10_a as ‘Text+Image’, and thus may setthe electronic device 100 to search for a query via at least one oftexts and images. Also, the user may set the settings bar 10_a as ‘All’,and thus may set the electronic device 100 to search for a query via atleast one of texts, images, and voices.

For example, if the user sets the settings bar 10_a as ‘Text’, theelectronic device 100 may generate the query, based on texts generatedby the user or texts provided to the user via a screen of the electronicdevice 100. For example, if the user sets the settings bar 10_a as‘Text+Image’, the electronic device 100 may generate the query, based ontexts and images generated by the user and/or texts and images providedto the user via the screen. The user input may be set through settings,other than the interface of the communication.

FIGS. 32 through 35 are flowcharts illustrating a method of providinginformation, the method being performed by the electronic device 100,according to exemplary embodiments.

Referring to FIG. 32, a processor of the electronic device 100 mayextract a plurality of keywords from at least one message displayed on ascreen of the electronic device 100 via a communication service (S210).The processor may determine priority orders of the keywords (S220). Theprocessor may generate a query, based on the priority orders of thekeywords, and may provide the query (S230). An input interface of theelectronic device 100 may receive an edited query from a user (S240).The processor may obtain information, based on the edited query andlocation information of the electronic device 100 (S250). An outputinterface of the electronic device 100 may provide the informationobtained by the processor (S260).

The method of providing information, the method being performed by theelectronic device 100, according to an exemplary embodiment maydetermine the priority orders of the keywords, and may generate thequery, based on the priority orders.

For example, as illustrated in FIG. 17, a plurality of keywords such as‘batman’, ‘Sumin’, and ‘L.A.’ may be extracted. For example, theelectronic device 100 may determine priority orders of the keywords,based on at least one of 1) content of conversation between users, 2) auser's location log, 3) user's previously input information, and 4)history information about a user. For example, a keyword having thehighest priority order from among the keywords may be generated as thequery.

A method of providing information, the method being performed by theelectronic device 100, according to another exemplary embodiment mayobtain information by using the query generated in operation S230. Also,a method of providing information, the method being performed by theelectronic device 100, according to another exemplary embodiment mayobtain information by using the edited query received in operation S240.Also, a method of providing information, the method being performed bythe electronic device 100, according to another exemplary embodiment mayobtain information by using the query generated in operation S230 andthe edited query received in operation S240, and may display theinformation. Also, a method of providing information, the method beingperformed by the electronic device 100, according to another exemplaryembodiment may obtain information by using the query generated inoperation S230 and the edited query received in operation S240, and maydisplay the information obtained in operation S250.

Referring to FIG. 33, the processor may extract a keyword from at leastone message displayed on the screen via the communication service(S310). The processor may generate a query related to the keyword, basedon user-related information, and may provide the query (S320). The inputinterface may receive an edited query from the user (S330). Theprocessor may obtain information, based on the edited query and locationinformation of the electronic device 100 (S340). The output interfacemay provide the information obtained by the processor (S350).

The method of providing information, the method being performed by theelectronic device 100, according to an exemplary embodiment may extractthe keyword or may generate the query, based on the user-relatedinformation. For example, the user-related information may include atleast one of a user's location log, user's previously input information,and history information about the user.

Referring to FIG. 34, the processor may extract a plurality of keywordsfrom at least one message displayed on the screen via the communicationservice (S410). The processor may generate and provide a query relatedto the keywords (S420). The input interface may receive an edited queryfrom the user (S430). The processor may obtain a plurality of pieces ofinformation, based on the edited query (S440). The output interface mayprovide the plurality of pieces of information obtained by the processor(S450).

According to the method of providing information, the method beingperformed by the electronic device 100, according to an exemplaryembodiment, the processor may obtain the plurality of pieces ofinformation, based on the edited query, and may provide the plurality ofpieces of information by considering priority orders of the plurality ofpieces of information. For example, the electronic device 100 maydetermine the priority orders of the plurality of pieces of information,based on user-related information. For example, the user-relatedinformation may include at least one of a user's location log, user'spreviously input information, and history information about the user.

Referring to FIG. 35, the processor may extract a plurality of keywordsfrom at least one message displayed on the screen via the communicationservice (S510). The processor may generate and provide a query relatedto the keywords (S520). The input interface may receive an edited queryfrom the user (S530). The processor may obtain a plurality of pieces ofinformation, based on the edited query and a current time (S540). Theoutput interface may provide the plurality of pieces of informationobtained by the processor (S550).

The method of providing information, the method being performed by theelectronic device 100, according to an exemplary embodiment may obtaininformation, based on an edited query and a current time. For example,if the current time is a lunch time of the user, the electronic device100 may obtain the information by prioritizing keywords related to arestaurant.

FIG. 36 illustrates an electronic device 1000, according to an exemplaryembodiment.

Referring to FIG. 36, the electronic device 1000 may providecommunication service Apps APP1, APP2, APP3, APP4, and APP₅ (also,referred to as the applications) to a user of the electronic device1000. The user may communicate with a user of another electronic device(not shown) by using a service provided by one of the communicationservice Apps APP1 through APP5.

For example, the communication service Apps APP1 through APP5 mayinclude a messaging service App, a mailing service App, and an SNS App.The messaging service App may include, for example, Microsoft network(MSN) messenger, Yahoo messenger, Nateon, Buddybuddy, KakaoTalk,KakaoStory, Skype, or the like. For example, the communication serviceApps APP1 through APP5 may include Facebook, Twitter, KakaoTalk, Skype,or the like.

For example, each of the communication service Apps APP1 through APP5may provide at least one of a one-to-one communication service, aone-to-many communication service, and a many-to-many communicationservice. For example, each of the communication service Apps APP1through APP5 may provide a one-to-one communication type messengerservice or a one-to-one communication type video call service. Forexample, each of the communication service Apps APP1 through APP5 mayprovide a one-to-many communication type video lecturing service. Forexample, each of the communication service Apps APP1 through APP5 mayprovide a many-to-many communication type video conference service.

For example, each of the communication service Apps APP1 through APP5may provide a voice or video communication service. For example, each ofthe communication service Apps APP1 through APP5 may provide acommunication service by using a 3D image. For example, each of thecommunication service Apps APP1 through APP5 may provide a communicationservice by using a 3D hologram.

The electronic device 1000 may include a query generation framework1100. The query generation framework 1100 may exchange information withthe communication service Apps APP1 through APP5 via an applicationprogramming interface (API).

The electronic device 1000 may generate a query by using the querygeneration framework 1100, without considering a type of an App used bya user. For example, the electronic device 1000 may generate the query,regardless of whether an App installed in the electronic device 1000 isthe mailing service App, a chatting service App, or the SNS App. Theelectronic device 1000 according to an exemplary embodiment may generatethe query any App installed therein by using the query generationframework 1100.

The query generation framework 1100 may extract at least one keywordfrom a message displayed on a screen of the electronic device 1000 via acommunication service. The query generation framework 1100 may extractthe at least one keyword from the message by using the natural languageprocessor 1200.

The query generation framework 1100 may provide a query related to theat least one keyword. For example, the query generation framework 1100may generate the query related to the at least one keyword by performinga statistical analysis and/or a semantic analysis.

For example, the query generation framework 1100 may generate ‘good daypizza’ as the query, based on the at least one keyword, by performingthe statistical analysis and/or the semantic analysis. For example, thequery generation framework 1100 may generate ‘good day pizza’ as thequery, based on 1) content of conversation between users, 2) a user'slocation log, 3) user's previously input information, and 4) historyinformation about the user.

The query generation framework 1100 may include the natural languageprocessor 1200. The natural language processor 1200 may recognize ameaning of the message by considering situational elements related tothe message. For example, the natural language processor 1200 of theelectronic device 1000 may recognize the meaning of the message, basedon types of words included in the message, a relation between the words,and meanings of the words. The natural language processor 1200 maydetermine a category of a search, based on the recognized meaning.

The electronic device 1000 may provide the user with the query generatedby using the query generation framework 1100. For example, theelectronic device 1000 may provide, via the screen, the user with animage as the query that is generated by using the query generationframework 1100. For example, the electronic device 1000 may provide, viaa speaker, the user with voice as the query that is generated by usingthe query generation framework 1100.

The electronic device 1000 may receive an edited query from the user.For example, the electronic device 1000 may provide the generated queryto the user, and may receive, via an interface, the query edited by theuser.

The electronic device 1000 may obtain information, based on the queryedited by the user and location information of the user. The electronicdevice 1000 may obtain the information from an external search server2000. For example, the electronic device 1000 may obtain the informationfrom the external search server 2000 via an App service. For example,the electronic device 1000 may obtain the information from the externalsearch server 2000 via a search service provided by an internet website.

Therefore, the electronic device 1000 according to an exemplaryembodiment may allow the user to conveniently receive informationrelated to ongoing communication while the user uses a communicationservice.

FIG. 37 is a block diagram illustrating the electronic device 1000,according to another exemplary embodiment.

Referring to FIG. 37, the electronic device 1000 may providecommunication service Apps APP1, APP2, APP3, APP4, and APP5 to a user ofthe electronic device 1000. The user may communicate with a user ofanother electronic device (not shown) by using a service provided by oneof the communication service Apps APP1 through APP5.

The electronic device 1000 may include the query generation framework1100. The query generation framework 1100 may exchange information withthe communication service Apps APP1 through APP5 via an API. The querygeneration framework 1100 may extract at least one keyword from amessage displayed on a screen of the electronic device 1000 via acommunication service.

The query generation framework 1100 may extract the at least one keywordfrom the message by using a natural language processor 3200 included ina cloud server 3000.

The query generation framework 1100 may provide a query related to theat least one keyword. For example, the query generation framework 1100may generate the query related to the at least one keyword by performinga statistical analysis and/or a semantic analysis.

The query generation framework 1100 may recognize a meaning of themessage by exchanging information with the natural language processor3200 included in the cloud server 3000. The natural language processor3200 may recognize the meaning of the message according to a situationrelated to the message. The natural language processor 3200 maydetermine a category of a search, based on the recognized meaning. Theelectronic device 1000 may provide the user with the query generated byusing the query generation framework 1100. The electronic device 1000may obtain information, based on a query edited by the user and locationinformation of the user.

Therefore, by using the external natural language processor 3200, theelectronic device 1000 according to an exemplary embodiment may allowthe user to conveniently receive information related to ongoingcommunication while the user uses a communication service.

FIG. 38 illustrates a configuration of the natural language processor1200 shown in FIG. 36 and the natural language processor 3200 shown inFIG. 37.

The natural language processor 1200 or 3200 may include a named entityrecognizer 1210 and a category classifier 1230. The named entityrecognizer 1210 may recognize a meaning of each of words used inmessages in communication. Also, the named entity recognizer 1210 mayrecognize a meaning of each of the messages. The category classifier1230 may classify categories of the messages, respectively. That is, thecategory classifier 1230 may check a keyword of each of the messages byanalyzing contents of each of the messages, and may classify thecategories of the messages, respectively.

FIG. 39 is a block diagram illustrating the electronic device 1000,according to another exemplary embodiment.

Referring to FIG. 39, the electronic device 1000 may providecommunication service Apps APP1, APP2, APP3, APP4, and APP5 to a user ofthe electronic device 1000. The user may communicate with a user ofanother electronic device (not shown) by using a service provided by oneof the communication service Apps APP1 through APP5.

The electronic device 1000 may include the query generation framework1100. The query generation framework 1100 may exchange information withthe communication service Apps APP1 through APP5 via an API. The querygeneration framework 1100 may extract at least one keyword from amessage displayed on a screen of the electronic device 1000 via acommunication service.

The electronic device 1000 may further include a GPS database 1400. AGPS module (or GPS processor) 1100-1 may continuously track a locationof the electronic device 1000 by using a location tracking engineincluded in the electronic device 1000. The GPS database 1400 maycontinuously accumulate data of the location tracked by the GPS module1100-1, and may continuously store information about the location of theelectronic device 1000.

The GPS module 1100-1 may receive a GPS signal from a GPS satellite, maydetermine the location of the electronic device 1000, and may generatelocation information. To do so, the GPS module 1100-1 may include a GPSantenna and a GPS receiver. The GPS module 1100-1 analyzes navigationdata included in the GPS signal, and extracts latitude and longitudedata, time information, or the like of the electronic device 1000. TheGPS signal includes an identification (ID) code, an orbit position, andtime information of the GPS satellite and the GPS module 1100-1calculates coordinates of the location of the electronic device 1000 bycalculating a difference between speeds of electromagnetic wavestransmitted from a plurality of GPS satellites. The aforementionedoperations may be performed by a GPS positioning program embedded in theGPS module 1100-1.

Here, the GPS positioning program may determine the location of theelectronic device 1000 by using one or more of conventional GPS (C-GPS)positioning, differential GPS (DGPS) positioning, assisted GPS (A-GPS)positioning, and double differential GPS positioning. Also, for moreprecise positioning according to the A-GPS positioning, the DGPSpositing, the double differential GPS positioning, etc., the GPSpositioning program may receive a signal related to error correctionfrom an external base station or the like, and may correct thepositioned location.

If the electronic device 1000 is a terminal that does not include theGPS module 1100-1, the electronic device 1000 may connect a separate GPSkit, which includes a GPS antenna, a GPS receiver, etc., to theelectronic device 1000, and may receive information about a locationpositioned by the GPS kit.

Therefore, the electronic device 1000 according to an exemplaryembodiment may continuously track the location of the electronic device1000, may continuously store location data, and thus may allow a user toconveniently receive information related to ongoing communication whilethe user uses a communication service.

FIG. 40 is a block diagram illustrating an operation of recognizing anentity, the operation being performed by the electronic device 1000,according to an exemplary embodiment.

Referring to FIG. 40, the natural language processor 1200 included inthe electronic device 1000 may include the named entity recognizer 1210and a frequency measurer 1250.

The frequency measurer 1250 may measure a frequency of use of recognizedentities in a conversation. For example, the named entity recognizer1210 may recognize <Peter Higgs> as an entity in a sentence [Who isPeter Higgs?]. The frequency measurer 1250 may count the number of timesthat <Peter Higgs>, which is recognized as the entity by the namedentity recognizer 1210, is used in the conversation between users.

The frequency measurer 1250 may calculate a rate of the frequency of useof the entity <Peter Higgs> in the conversation between the users. Forexample, the named entity recognizer 1210 may also recognize <NobelPrize> as an entity. When the entity <Nobel Prize> is used seven timesin the conversation, and the entity <Peter Higgs> is used fourteen timesin the conversation, the frequency measurer 1250 may calculate thefrequency of the entity <Peter Higgs> as 0.67. The frequency measurer1250 may calculate a usage frequency by dividing <the number of timesthat a particular entity is used> by <a total number of times all of theentities are used>.

The named entity recognizer 1210 may receive information about each ofthe entities from the cloud server 3000.

The cloud server 3000 may include a data computator 3400. The datacomputator 3400 may include a text extractor 3410 and a data storage3430. The data storage 3430 may include various types of databases suchas a cloud-named-entity recognition database (cloud NER DB) 3431 and aknowledge DB 3433.

The text extractor 3410 may mine a text from the knowledge DB 3433stored in the data storage 3430. The text extractor 3410 may store mineddata in the cloud NER DB 3431 of the cloud server 3000. The datacomputator 3400 may arrange information about the cloud NER DB 3431,according to a table chart shown in FIG. 41, and may store theinformation in the data storage 3430.

The electronic device 1000 according to an exemplary embodiment mayinclude the named entity recognizer 1210, may recognize each of entitiesincluded in one or more messages in a conversation between users, andmay count a frequency that each of the entities is used. Therefore, theelectronic device 1000 may extract a keyword, may generate a query, ormay limit extraction of a keyword or a query, by using entityinformation such as a frequency that each of the entities is used in theconversation.

FIG. 41 illustrates a table chart generated by the named entityrecognizer 1210 after the named entity recognizer 1210 recognizesentities, according to an exemplary embodiment.

Referring to FIG. 41, the named entity recognizer 1210 of FIG. 40 mayrecognize <Peter Higgs>, <European Center for Nuclear Research>, <UnitedStates>, and <Switzerland> as the entities. The named entity recognizer1210 may extract information about each of the entities (hereinafter,referred to as the entity information) from the data storage 3430included in the cloud server 3000. For example, the entity informationmay include relevance, a sentiment, a type, and linked data.

The electronic device 1000 may extract a keyword or may generate a queryby using the entity information. Therefore, the electronic device 1000uses the entity information in a natural language analysis, so that theelectronic device 1000 may further clearly recognize user's intentionincluded in a message.

FIG. 42 is a block diagram illustrating an operation of obtaining a userinput, the operation being performed by the electronic device 1000,according to an exemplary embodiment.

Referring to FIG. 42, a user context awareness framework 1500 includedin the electronic device 1000 may include a text assistance part 1510,an image assistance part 1520, and a voice assistance part 1530. Also,the user context awareness framework 1500 may include a service binder1550. The text assistance part 1510 may include a context aware textextractor 1513 and a query generator 1515.

The service binder 1550 may receive the user input, and may deliver theuser input to one of the text assistance part 1510, the image assistancepart 1520, and the voice assistance part 1530, according to an inputtype. Also, the service binder 1550 may receive a generated query fromthe text assistance part 1510, and may deliver the generated query to auser.

The text assistance part 1510 may generate a query with respect to atext-type user input received via the service binder 1550. The textassistance part 1510 may continuously mine texts by using the contextaware text extractor 1513, and may generate the query by using the querygenerator 1515. The generated query may be connected to the searchserver 2000 or the cloud server 3000 and thus may be connected to asearch engine 3500, an image analysis engine 3600, or a voice assistanceengine 3700.

The image assistance part 1520 may receive an image and may generate atext corresponding to the received image. The generated text may betransmitted to the query generator 1515, and then the query generator1515 may generate a query in a same manner as the text-type user input.

The voice assistance part 1530 may receive voice and may generate a textcorresponding to the received voice. The generated text may betransmitted to the query generator 1515, and then the query generator1515 may generate a query in a same manner as the text-type user input.

The text assistance part 1510, the image assistance part 1520, and thevoice assistance part 1530 may be activated or inactivated via theinterface shown in FIG. 31B. For example, when the user sets thesettings bar 10_a as ‘Text’ by using the interface of FIG. 31B, theimage assistance part 1520 and the voice assistance part 1530 may beinactivated, and even if the electronic device 1000 receives an imageinput or a voice input, the electronic device 1000 may not generate aquery corresponding to the image input or the voice input.

FIG. 43 is a block diagram illustrating an operation of obtaining a userinput, the operation being performed by the electronic device 1000,according to another exemplary embodiment.

Referring to FIG. 43, the electronic device 1000 may include a searchengine 1850, an image analysis engine 1860, and/or a voice assistanceengine 1870. That is, unlike the exemplary embodiment of FIG. 42, thesearch engine 1850, the image analysis engine 1860, and/or the voiceassistance engine 1870 may be included in the electronic device 1000.The electronic device 1000 may use the search engine 1850, the imageanalysis engine 1860, or the voice assistance engine 1870 to receive aquery and to obtain information.

FIG. 44 is a block diagram illustrating a configuration of a userterminal device 1000, according to an exemplary embodiment. The userterminal device 1000 shown in FIG. 44 may correspond to the electronicdevice 100 of FIG. 1.

As illustrated in FIG. 44, the configuration of the user terminal device1000 may be applied to various types of devices including a mobilephone, a tablet PC, PDA, an MP3 player, a kiosk, an electronicphoto-frame, a navigation device, a digital television (TV), or wearabledevices including a wristwatch, a HMD, or the like.

Referring to FIG. 44, the user terminal device 1000 may include at leastone of a display 1100-2, a controller 1700, a memory 1200, a GPS chip1250, a communicator 1300, a video processor 1350, an audio processor1400, a user input device 1450, a microphone 1500, an image capturer1550, a speaker 1600, and a motion detector 1650.

The display 1100-2 may include a display panel 1110 and a controller(not shown) that controls the display panel 1110. The display panel 1110may be embodied as various displays including a liquid crystal display(LCD), an organic light-emitting diode (OLED) display, an active matrixOLED (AMOLED) display, a plasma display panel (PDP), or the like. Thedisplay panel 1110 may be formed to be flexible, transparent, and/orwearable. The display panel 1110 may be combined with a touch panel 1470of the user input device 1450, and thus may be provided as a touchscreen(not shown). For example, the touchscreen may include an integratedmodule having a stack structure containing the display panel 1110 andthe touch panel 1470.

The memory 1200 may include at least one of an internal memory (notshown) and an external memory (not shown).

The internal memory may include at least one of a volatile memory (e.g.,a dynamic random-access memory (DRAM), a static random access memory(SRAM), a synchronous dynamic random access memory (SDRAM), etc.), anon-volatile memory (e.g., a one-time programmable read-only memory(OTPROM), a programmable read only memory (PROM), an erasable andprogrammable read only memory (EPROM), an electrically erasable andprogrammable read only memory (EEPROM), a mask read only memory (ROM), aflash ROM, etc.), a hard disk drive (HDD), and a solid-state drive(SSD). According to an exemplary embodiment, the controller 1700 mayload a command or data, which is received from at least one of thenon-volatile memory and another element, to the volatile memory, and mayprocess the command or the data. Also, the controller 1700 may store, inthe non-volatile memory, data that is received from or is generated byanother element.

The external memory may include at least one of a compact flash (CF)memory, a secure digital (SD) memory, a micro secure digital (micro-SD)memory, a mini secure digital (mini-SD) memory, an extreme digital (xD)memory, and a memory stick.

The memory 1200 may store various programs and data used in operationsof the user terminal device 1000. For example, the memory 1200 maytemporarily or semi-permanently store a portion of contents to bedisplayed on a lock screen.

The controller 1700 may control the display 1100-2 to display theportion of the contents which is stored in the memory 1200. In otherwords, the controller 1700 may display, on the display 1100-2, theportion of the contents which is stored in the memory 1200.Alternatively, when a user's gesture is performed in a region of thedisplay 1100-2, the controller 1700 may perform a control operation thatcorresponds to the user's gesture.

The controller 1700 may include at least one of a random access memory(RAM) 1701, a ROM 1720, a central processing unit (CPU) 1730, a graphicsprocessor unit (GPU) 1740, and a bus 1750. The RAM 1710, the ROM 1720,the CPU 1730, and the GPU 1740 may be connected to each other via thebus 1750.

The CPU 1730 may access the memory 1200, and performs a bootingoperation by using an operating system (OS) stored in the memory 1200.Also, the CPU 1730 performs various operations by using the variousprograms, a plurality of contents, a plurality of pieces of data, etc.stored in the memory 1200.

The ROM 1720 stores a command set for booting up a system. For example,when a turn-on command is input to the user terminal device 1000, andpower is supplied to the user terminal device 1000, the CPU 1730 maycopy the OS stored in the memory 1200 to the RAM 1710, according to thecommand stored in the ROM 1720, may execute the OS, and thus may boot upthe system. When the booting operation is completed, the CPU 1730 copiesthe various programs stored in the memory 1200 to the RAM 1710, andperforms the various operations by executing the programs copied to theRAM 1710. When the user terminal device 1000 is booted up, the GPU 1740displays a user interface screen in a region of the display 1100-2. Inmore detail, the GPU 1740 may generate a screen that displays anelectronic document including various objects such as content, an icon,a menu, or the like. The GPU 1740 calculates coordinate values of theobjects that are to be displayed according to a layout of the userinterface screen, and calculates attribute values of shapes, sizes, orcolors of the objects. Then, the GPU 1740 may generate user interfacescreens with various layouts including the objects based on thecalculated attribute values. The user interface screen generated by theGPU 1740 may be provided to the display 1100-2 and thus may be displayedin regions of the display 1100-2.

The GPS chip 1250 may receive a GPS signal from a GPS satellite and maycalculate a current position of the user terminal device 1000. In a casewhere a navigation program is used or a current position of the user isneeded, the controller 1700 may calculate a position of the user byusing the GPS chip 1250.

The communicator 1300 may perform communication with various externaldevices according to various types of communication methods. Thecommunicator 1300 may include at least one selected from a Wi-Fi chip1310, a Bluetooth chip 1320, a wireless communication chip 1330, and anear field communication (NFC) chip 1340. The controller 1700 mayperform the communication with the various external devices by using thecommunicator 1300.

The Wi-Fi chip 1310 and the Bluetooth chip 1320 may performcommunication by using Wi-Fi and Bluetooth, respectively. If the Wi-Fichip 1310 or the Bluetooth chip 1320 is used, the Wi-Fi chip 1310 or theBluetooth chip 1320 may first transmit and receive various types ofconnection information including a service set identification (SSID), asession key, or the like, may establish a connection for communicationby using the connection information, and then may transmit and receivevarious types of information. The wireless communication chip 1330 mayindicate a chip that performs communication according to variouscommunication standards such as the Institute of Electrical andElectronics Engineers (IEEE), ZigBee, 3rd generation (3G), 3rdGeneration Partnership Project (3GPP), Long Term Evolution (LTE), or thelike. The NFC chip 1340 indicates a chip that operates in using NFC byusing a 13.56 MHz band from among various radio frequency-identification(RF-ID) frequency bands such as 135 kHz, 13.56 MHz, 433 MHz, 860˜960MHz, 2.45 GHz, or the like.

The video processor 1350 may process video data included in contentreceived by using the communicator 1300 or may process video dataincluded in content stored in the memory 1200. The video processor 1350may perform various image processing such as decoding, scaling, noisefiltering, frame rate conversion, resolution conversion, or the like onthe video data.

The audio processor 1400 may process audio data included in contentreceived by using the communicator 1300 or may process audio dataincluded in content stored in the memory 1200. The audio processor 1400may perform various processing such as decoding, amplification, noisefiltering, or the like on the audio data.

When a reproducing program for multimedia content is executed, thecontroller 1700 may reproduce the multimedia content by driving thevideo processor 1350 and the audio processor 1400. The speaker 1600 mayoutput audio data generated in the audio processor 1400.

The user input device 1450 may receive an input of various instructionsfrom a user. The user input device 1450 may include at least oneselected from a key 1460, a touch panel 1470, and a pen recognizingpanel 1480.

The key 1460 may be of various types such as a mechanical button, awheel, or the like that may be formed in a front portion, a sideportion, a rear portion, etc., of an external surface of a body of theuser terminal device 1000.

The touch panel 1470 may sense a touch input by the user and may outputa value of a touch event that corresponds to a signal generated by thesensed touch input. When the touch panel 1470 is combined with thedisplay panel 1110 and thus is formed as a touchscreen, the touchscreenmay be configured as, for example, a capacitive touchscreen, a resistivetouchscreen, or a piezoelectric touchscreen by using various types oftouch sensors. The capacitive touchscreen may calculate touchcoordinates by sensing a small amount of electricity generated when abody part of the user touches the surface of the capacitive touchscreen,which is coated with a dielectric material. The resistive touchscreenmay include two embedded electrode plates and may calculate touchcoordinates by sensing a flow of current that occurs when the usertouches the resistive touchscreen which causes upper and lower plates ofa touched point to contact each other. The touch event that occurs onthe touchscreen may be mainly generated by a finger of a person but mayalso be generated by an object formed of a conductive material capableof changing capacitance.

The pen recognizing panel 1480 may sense a proximity input or a touchinput of a touch pen (e.g., a stylus pen or a digitizer pen) which isperformed by a user, and may output a sensed pen proximity event or asensed pen touch event. The pen recognizing panel 1480 may include, forexample, an electromagnetic resonance (EMR)-type pen recognizing panel,and may sense the touch input or the proximity input according tochanges in a strength of an electromagnetic field, which occur when thetouch pen approaches or touches the touchscreen. In more detail, the penrecognizing panel 1480 may include an electromagnetic induction coilsensor (not shown) having a grid structure, and an electric signalprocessor (not shown) for sequentially providing an alternating current(AC) signal having a predetermined frequency to each loop coil of theelectromagnetic induction coil sensor. When a pen having an internalresonance circuit is positioned near a loop coil of the pen recognizingpanel 1480, a magnetic field transmitted from the loop coil generates acurrent in the resonance circuit in the pen, based on mutualelectrostatic induction. Due to the current, an induction field isgenerated from a coil forming the resonance circuit in the pen, and thepen recognizing panel 1480 detects the induction field from the loopcoil capable of receiving a signal, and thus senses the touch input orthe proximity input by the pen. The pen recognizing panel 1480 may bearranged to occupy a preset area below the display panel 1110, e.g., mayhave a size capable of covering a display region of the display panel1110.

The microphone 1500 may receive an input of a user's voice or othersound and may convert the user's voice or other sound to audio data. Thecontroller 1700 may use the user's voice, which is input via themicrophone 1500, in a call-related operation or may convert the user'svoice to the audio data and may store the audio data in the memory 1200.

The image capturer 1550 may capture a still image or a moving pictureaccording to a control by the user. The image capturer 1550 may beplural in number and include a front camera, a rear camera, or the like.

If the image capturer 1550 and the microphone 1500 are provided, thecontroller 1700 may perform a control operation according to a user'svoice input via the microphone 1500 or a user's motion recognized by theimage capturer 1550. For example, the user terminal device 1000 mayoperate in a motion control mode or a voice control mode. If the userterminal device 1000 operates in the motion control mode, the controller1700 may activate the image capturer 1550 and may capture an image ofthe user, may trace a change in motions of the user, and may perform acontrol operation corresponding thereto. If the user terminal device1000 operates in the voice control mode (i.e., a voice recognitionmode), the controller 1700 may analyze a user's voice input via themicrophone 1500, and may perform a control operation according to theanalyzed user's voice.

The motion detector 1650 may detect movement of a body of the userterminal device 1000. The user terminal device 1000 may rotate or maytilt in various directions. Here, the motion detector 1650 may detect amovement characteristic such as a rotation direction, a rotation angle,a tilted angle, or the like by using at least one of a magnetic sensor,a gyroscope sensor, an acceleration sensor, etc.

In addition, an exemplary embodiment may further include a universalserial bus (USB) port for connecting the user terminal device 1000 and aUSB connector, various external input ports including a headset, amouse, a local area network (LAN), etc. for connection with variousexternal terminals, a digital multimedia broadcasting (DMB) chip forreceiving and processing a DMB signal, various sensors, or the like.

Names of elements in the user terminal device 1000 may vary. Also, theuser terminal device 1000 according to an exemplary embodiment mayinclude at least one from among the aforementioned elements, or may beembodied with more or less elements than the aforementioned elements.

FIG. 45A is a block diagram illustrating a software configuration of theuser terminal device 1000, according to an exemplary embodiment.

Referring to FIG. 45A, the memory 1200 may store an OS for controllingresources of the user terminal device 1000, an application program foran operation of an application, or the like. The OS may include a kernel12100, middleware 1220, an API 12300, etc. Examples of the OS mayinclude Android, iOS, Windows, Symbian, Tizen, Bada or the like.

The kernel 12100 may include at least one of a device driver 1210-1 anda system resource manager 1210-2 that are capable of managing resources.The device driver 1210-1 may allow hardware of the user terminal device1000 to be accessed and controlled by using software. To do so, thedevice driver 1210-1 may be divided into an interface and discretedriver modules provided by discrete hardware companies. For example, thedevice driver 1210-1 may include at least one of a display driver, acamera driver, a Bluetooth driver, a shared memory driver, an USBdriver, a keypad driver, a Wi-Fi driver, an audio driver, and aninter-process communication (IPC) driver. The system resource manager1210-2 may include at least one of a process manager, a memory manager,and a file system manager. The system resource manager 1210-2 mayperform controlling, allocating, or retrieving of system resources.

The middleware 1220 may include a plurality of modules that arepre-embodied to provide a function that is commonly requested by variousapplications. The middleware 1220 may provide the function via the API12300 to control applications 1240 to efficiently use the resources ofthe user terminal device 1000. The middleware 1220 may include at leastone of a plurality of modules including an application manager 1220-1, aWindows manager 1220-2, a multimedia manager 1220-3, a resource manager1220-4, a power manager 1220-5, a DB manager 1220-6, a package manager1220-7, a connection manager 1220-8, a notification manager 1220-9, alocation manager 1220-10, a graphics manager 1220-11, and a securitymanager 1220-12.

The application manager 1220-1 may manage a life cycle of at least oneof the applications 1240. The Windows manager 1220-2 may manage a GUIresource used in a screen. The multimedia manager 1220-3 may detectformats for reproducing various types of multimedia files, and mayencode or decode a media file by using a codec appropriate for a formatof the media file. The resource manager 1220-4 may manage resources suchas a source code, a memory, or storage spaces of at least one of theapplications 1240. The power manager 1220-5 may interoperate with aBasic Input/Output System (BIOS), thereby managing a battery or powersupply and providing power information to an operation. The DB manager1220-6 may manage a database, which is to be used in at least one of theapplications 1240, to be generated, searched, or changed. The packagemanager 1220-7 may manage installing or updating an application that isdistributed in the form of a package file. The connection manager 1220-8may manage wireless connection such as Wi-Fi, Bluetooth, or the like.The notification manager 1220-9 may display or notify an event such as areceived message, an appointment, an approach notification, or the likewhile a user is not disturbed by the event. The location manager 1220-10may manage location information of the user terminal device 1000. Thegraphics manager 1220-11 may manage graphics effects to be provided tothe user, and a UI related to the graphics effects. The security manager1220-12 may provide general security functions requested for systemsecurity or user authentication. If the user terminal device 1000 has aphone function, the middleware 1220 may further include a call manager(not shown) for managing a voice or video call function of the userterminal device 1000.

The middleware 1220 may further include a runtime library 1220-13 orother library modules (not shown). The runtime library 1220-13 is alibrary module used by a compiler to add a new function by using aprogramming language while an application is being executed. Forexample, the runtime library 1220-13 may perform input/output, memorymanagement, or an operation for performing a mathematical function. Themiddleware 1220 may generate and use a new middleware module bycombining various functions of the aforementioned internal configurationelement modules. The middleware 1220 may provide a module that isspecialized according to each type of the OS, to provide adifferentiated function. The middleware 1220 may dynamically delete someof existing configuration elements or may dynamically add newconfiguration elements. Some configuration elements in an exemplaryembodiment may be omitted, other configuration elements may be furtherincluded, or some of the configuration elements may be substituted withother configuration elements that have different names but performsimilar functions.

The API 12300 is a set of API programming functions, and may be providedwith a different configuration according to each type of OS. Android oriOS may provide one API set for each platform, and Tizen may provide atleast two API sets for each platform.

The applications 1240 may include at least one pre-loaded applicationthat is installed as a default, or a third party application that isinstalled and used by the user while the user uses the user terminaldevice 1000. For example, the applications 1240 may include at least oneof a home application 1240-1 for returning to a home screen, a dialerapplication 1240-2 for establishing a call with the other party, a textmessage application 1240-3 for receiving a message from the other partyidentified based on a telephone number, an instant message (IM)application 1240-4, a browser application 1240-5, a camera application1240-6, an alarm application 1240-7, a phone-book application 1240-8 formanaging telephone numbers or addresses of other parties, a call logapplication 1240-9 for managing a call log of the user,reception/transmission logs of text messages, or an unanswered call log,an e-mail application 1240-10 for receiving an e-mail from another partyidentified based on an e-mail account, a calendar application 1240-11, amedia player application 1240-12, an album application 1240-13, and aclock application 1240-14. Names of the aforementioned elements ofsoftware according to an exemplary embodiment may vary according to eachtype of OS. Also, the software according to an exemplary embodiment mayinclude or omit at least one of the aforementioned elements, or mayfurther include at least one additional element.

The applications 1240 may receive a query via the middleware 1220 andthe API 12300. When a keyword is extracted by performing a naturallanguage analysis while an application is being executed, the userterminal device 1000 may determine a query corresponding to the keywordvia the API 12300 and the middleware 1220. For example, when a keyword“6th of a certain month” is extracted while a user executes the textmessage application 1240-3, the user terminal device 1000 may search forschedule data generated in the calendar application 1240-11 via the API12300 and the middleware 1220, and may display a search result on anexecution window of the text message application 1240-3.

Also, when a keyword “execute xx” is extracted while the user executesthe text message application 1240-3, the user terminal device 1000 maysearch for data corresponding to “execute xx” in the calendarapplication 1240-11, the phone-book application 1240-8, and the e-mailapplication 1240-10 via the API 12300 and the middleware 1220, and maydisplay a search result on the execution window of the text messageapplication 1240-3.

The user terminal device 1000 may deliver the extracted keyword to themiddleware 1220 via a dedicated API, may access, via the DB manager1220-6, data generated in another application, and may provide thesearch result by referring to the data.

A plurality of pieces of data generated in the applications 1240 may bestored in a DB (not shown), and when the keyword is extracted while anapplication is being executed, the user terminal device 1000 may accessthe DB via the API 12300 and the middleware 1220, and may provide thesearch result by referring to data stored in the DB.

FIG. 45B illustrates a user settings screen provided by the electronicdevice 100, according to an exemplary embodiment.

Referring to FIG. 45B, the electronic device 100 may provide a user witha settings screen 4501 with respect to an information providingoperation. The user may select an information providing method via thesettings screen 4501. While FIG. 45B illustrates an operation of “QueryGeneration”, this operation may be differently expressed according toproducts, and thus the expression of the operation does not limit thescope of the inventive concept.

The query generation may provide three types, i.e., “App 4503”, “Device4505”, and “ALL 4507”. When the user selects “App 4503” displayed on thesettings screen 4501, the electronic device 100 may generate a query byconsidering only an executed application, and may provide a searchresult performed by using the query. For example, when the user selects“App 4503” from the settings screen 4501, and executes the text messageapplication 1240-3, the electronic device 100 may extract a keyword byperforming a natural language analysis, may determine a query byconsidering only the text message application 1240-3, and may perform asearch operation.

When the user selects “Device 4505” from the settings screen 4501, theelectronic device 100 may determine a query by considering datagenerated in an executed application and another application stored in aDB (not shown), and may perform a search operation. For example, whenthe user selects “Device 4505” from the settings screen 4501, andexecutes the text message application 1240-3, the electronic device 100may extract a keyword by performing a natural language analysis, maydetermine a query corresponding to the keyword by considering thecalendar application 1240-11, the phone-book application 1240-8, and thee-mail application 1240-10 via the API 12300 and the middleware 1220,and may display a search result on an execution window of the textmessage application 1240-3.

The electronic device 100 may deliver the extracted keyword to themiddleware 1220 via a dedicated API, may access, via the DB manager1220-6, data generated in another application, and may provide thesearch result by referring to the data.

A plurality of pieces of data generated in the applications 1240 may bestored in the DB, and when the keyword is extracted while an applicationis being executed, the electronic device 100 may access the DB via theAPI 12300 and the middleware 1220, and may provide the search result byreferring to data stored in the DB.

When the user selects “ALL 4507” from the settings screen 4501, theelectronic device 100 may select all of “App 4503” and “Device 4505”,and may perform a search operation.

FIG. 46 illustrates a UI of an electronic device 20000, according toanother exemplary embodiment. The electronic device 20000 may correspondto the electronic device 100 of FIG. 1.

The electronic device 20000 may include a processor 2700, an inputinterface 2400, and an output interface 2100.

The processor 2700 may include a mobile application processor or a CPU.The processor 2700 may be referred to as a controller, a controller, orthe like. Also, the processor 2700 may further include a core, a displaycontroller, and an image signal processor (ISP).

The processor 2700 according to an exemplary embodiment may extract atleast one keyword from a message displayed on a screen of the electronicdevice 20000 via a communication service. Also, the processor 2700according to an exemplary embodiment may generate a query related to thekeyword. Also, the processor 2700 according to an exemplary embodimentmay obtain information, based on an edited query and locationinformation of the electronic device 20000.

The input interface 2400 may be a unit used by a user to input data tocontrol the electronic device 20000. For example, the input interface2400 may be a key pad, a dome switch, a touch pad (for example, a touchcapacitive type touch pad, a pressure resistive type touch pad, aninfrared beam sensing type touch pad, a surface acoustic wave type touchpad, an integral strain gauge type touch pad, a piezo effect type touchpad, etc.), a jog wheel, a jog switch, or the like. Also, the inputinterface 2400 may include a touchscreen, a touch panel, a microphone,and/or a keyboard.

Also, the input interface 2400 may include at least one module toreceive an input of data from the user. For example, the input interface2400 may include a motion recognition module, a touch recognitionmodule, a voice recognition module, etc.

The touch recognition module may sense a touch gesture by the user onthe touchscreen, and may deliver information about the touch gesture tothe processor 2700. The voice recognition module may recognize voice ofthe user by using a voice recognition engine, and may deliver therecognized voice to the processor 2700. The motion recognition modulemay recognize a motion of an object, and may deliver information aboutthe motion of the object to the processor 2700.

Throughout the specification, a user input via the input interface 2400of the electronic device 20000 may include, but is not limited to, atleast one selected from a touch input, a bending input, a voice input, akey input, and a multimodal input.

The touch input indicates a gesture of the user which is performed on atouch screen to control the electronic device 20000. For example, thetouch input according to one or more exemplary embodiments may include atap gesture, a touch & hold gesture, a double tap gesture, a draggesture, a panning gesture, a flick gesture, a drag & drop gesture, orthe like.

“Tapping” is a user's motion of touching a screen by using a finger or atouch tool (e.g., an electronic pen) and then instantly lifting thefinger or touch tool from the screen.

“Touching & holding” is a user's motion of touching a screen by using afinger or a touch tool (e.g., an electronic pen) and then maintainingthe above touching motion for a predetermined time (e.g., 2 seconds) orlonger, after touching the screen. For example, a time differencebetween a touch-in time and a touch-out time is greater than or equal tothe predetermined time (e.g., 2 seconds). When a touch input lasts morethan the predetermined time, in order to inform the user whether thetouch input is tapping or touching & holding, a feedback signal may beprovided in a visual, acoustic, or tactile manner. In other exemplaryembodiments, the predetermined time may vary.

“Double tapping” is a user's motion of rapidly touching the screen twiceby using a finger or touch tool (such as a stylus pen).

“Dragging” is a user's motion of touching a screen by using the fingeror touch tool and moving the finger or touch tool to another position onthe screen while keeping the touching motion. The dragging motion mayenable the moving or panning motion of an object.

“Panning” is a user's motion of performing a dragging motion withoutselecting an object. Since no object is selected in the panning motion,no object is moved in a page but the page itself is moved on the screenor a group of objects may be moved within a page.

“Flicking” is a user's motion of rapidly performing a dragging motionover a predetermined speed (e.g., 100 pixel/s) by using the finger ortouch tool. The dragging (panning) motion or the flicking motion may bedistinguished from each other based on whether a moving speed of thefinger or touch tool is over the predetermined speed (e.g., 100 pixel/s)or not.

“Dragging & Dropping” is a user's motion of dragging an object to apreset position on the screen with the finger or touch tool and thendropping the object at that position.

“Pinching” is a user's motion of moving two fingers touching the screenin opposite directions. The pinching motion is a gesture to magnify(open pinch) or contract (close pinch) an object or a page. Amagnification value or a contraction value is determined according tothe distance between the two fingers.

“Swiping” is a user's motion of touching an object on the screen withthe finger or touch tool and simultaneously moving the objecthorizontally or vertically by a predetermined distance. A swiping motionin a diagonal direction may not be recognized as a swiping event.

“Motion input” indicates a motion that a user does with the device tocontrol the electronic device 20000. For example, the motion input mayinclude an input of the user who rotates the electronic device 20000,tilts the electronic device 20000, or moves the electronic device 20000in up, down, right, and/or left direction. The electronic device 20000may sense a motion input preset by the user, by using an accelerationsensor, a tilt sensor, a gyro sensor, a 3-axis magnetic sensor, etc.

Throughout the specification, “bending input” indicates an input of auser who bends a whole or partial area of the electronic device 20000 tocontrol the electronic device 20000, and here, the electronic device20000 may be a flexible display device. According to an exemplaryembodiment, the electronic device 20000 may sense a bending position (acoordinates-value), a bending direction, a bending angle, a bendingspeed, the number of times that the bending motion is performed, a timeof occurrence of the bending motion, a hold time of the bending motion,etc.

Throughout the specification, “key input” indicates an input of the userwho controls the electronic device 20000 by using a physical key formedon the electronic device 20000 or by using a virtual key provided by theelectronic device 20000.

Throughout the specification, “multimodal input” indicates a combinationof at least two input methods. For example, the electronic device 20000may receive a touch input and a motion input of the user, or may receivea touch input and a voice input of the user. Also, the electronic device20000 may receive a touch input and an eye input of the user. The eyeinput indicates an input by which the user adjusts a blinking motion ofhis or her eye, a gaze position, a moving speed of his or her eye, etc.to control the electronic device 20000.

The input interface 2400 according to an exemplary embodiment mayreceive an edited query from the user.

The output interface 2100 may output an audio signal, a video signal, oran alarm signal, and may include a display module, a sound outputmodule, or the like. Also, the output interface 2100 may include a flatdisplay capable of displaying a 2D image, and a flat display capable ofdisplaying a 3D image. The output interface 2100 may include a devicecapable of outputting a 3D hologram.

The output interface 2100 according to an exemplary embodiment mayprovide the query generated by the processor 2700. The output interface2100 according to an exemplary embodiment may provide informationobtained by the processor 2700.

The electronic device 20000 may exchange information with a searchserver (not shown) via communication with the search server. Forexample, the electronic device 20000 may communicate with the searchserver by using at least one protocol. For example, the electronicdevice 20000 may communicate with the search server by using at leastone of a trivial file transfer protocol (TFTP), a simple networkmanagement protocol (SNMP), a simple mail transfer protocol (SMTP), apost office protocol (POP), an internet control message protocol (ICMP),a serial line interface protocol (SLIP), a point-to-point protocol(PPP), a dynamic host configuration protocol (DHCP), a network basicinput/output system (NetBIOS), an internetwork packet exchange/sequencedpacket exchange (IPX/SPX), an Internet control message protocol (ICMP),an Internet protocol (IP), an address resolution protocol (ARP), atransmission control protocol (TCP), a user datagram protocol (UDP),Winsock, a dynamic host configuration protocol (DHCP), and a routinginformation protocol (RIP), but one or more exemplary embodiments arenot limited thereto.

The electronic device 20000 may short-distance communication by using ashort-distance communication module. Examples of the short-distancecommunication may include a wireless LAN (e.g., Wi-Fi), Bluetooth,ZigBee, Wi-Fi direct (WFD), UWB, or infrared data association (IrDA),but one or more exemplary embodiments are not limited thereto.

FIGS. 47 through 50 illustrate examples of a UI provided by theelectronic device 100 for a communication via a messaging service.

Referring to FIG. 47, the electronic device 100 may time-sequentiallydisplay, on a display 4701, messages received from another electronicdevice, and messages transmitted to the other electronic device. Areference numeral 4703 of FIG. 47 indicates a virtual screen that showsmagnified and extended view of messages displayed on the display 4701.Only a portion of the virtual screen 4703 is displayed on the display4701. That is, when the electronic device 100 time-sequentially displaysthe messages, the electronic device 100 may display apreviously-displayed message on an upper area of the display 4701, andthe time-sequentially displayed messages may be shown on the display4701 while being scrolled.

While the electronic device 100 continuously monitors content ofconversation between users, the electronic device 100 may separatelyindicate or may separately mark a keyword included in the messages, andmay maintain the indication or the mark for a preset time period. Theelectronic device 100 may extract at least one keyword from themessages, and may determine a query of a search service, based on theextracted at least one keyword and context information of the user. Theelectronic device 100 may extract the at least one keyword by performinga natural language analysis on contents of the messages. The electronicdevice 100 may extract a keyword by using a previous message and acurrent message displayed on the display 4701. Also, the electronicdevice 100 may use the extracted keyword as a query of the searchservice, or may deduce another keyword from the extracted keyword andmay use the deduced keyword as a query of the search service.

The display 4701 may differently mark the keyword to be distinguishablefrom other contents of the displayed messages, wherein the keyword isused in determining the query.

On the virtual screen 4703 of FIG. 47, keywords 4711, 4713, 4715, 4717,and 4719 of communication messages are marked to be distinguishable fromother contents of the communication messages. The keyword may bedifferently (or distinguishably) marked from other contents for a presettime period, e.g., for one hour. That is, the keyword that isdifferently marked may return to its originally-displayed state after anelapse of the preset time period. The keyword may return to itsoriginally-displayed state after one hour, and may be equally displayedas the other contents. Alternatively, a time period during which thekeyword is differently marked from the other contents may be determined,according to a user input. For example, if a user sets two hours as thetime period during which the keyword is differently marked from theother contents of the communication messages, the keyword may bedifferently marked from the other contents for two hours.

Referring to FIG. 48, the electronic device 100 may time-sequentiallydisplay, on a display 4801, messages received from another electronicdevice, and messages transmitted to the other electronic device. Areference numeral 4803 of FIG. 48 indicates a virtual screen that showsa magnified and extended view of messages displayed on the display 4801.Only a portion of the virtual screen 4803 is displayed on the display4801. That is, when the electronic device 100 time-sequentially displaysthe messages, the electronic device 100 may display apreviously-displayed message on an upper area of the display 4801, andthe time-sequentially displayed messages may be shown on the display4801 while being scrolled.

While the electronic device 100 continuously monitors content ofconversation between users, the electronic device 100 may separatelyindicate or may separately mark a keyword included in the messages onthe display 4801, and may maintain the indication or the mark for apreset time period. The electronic device 100 may extract at least onekeyword by using one or more keywords included in the messages, and maydetermine a query of a search service, based on the extracted at leastone keyword and context information of the user. The electronic device100 may extract the at least one keyword by performing a naturallanguage analysis on contents of the messages. The electronic device 100may extract a keyword by using a previous message and a current messagedisplayed on the display 4801. Also, the electronic device 100 may usethe extracted keyword as a query of the search service, or may deduceanother keyword from the extracted keyword and may use the deducedkeyword as a query of the search service.

The display 4801 may differently mark the keyword to be distinguishablefrom other contents of the displayed messages, wherein the keyword isused in determining the query.

On the virtual screen 4803 of FIG. 48, keywords 4811, 4813, 4815, 4817,and 4819 of communication messages are differently marked from othercontents of the communication messages. The keyword may be differentlymarked from other contents for a preset time period, e.g., for one hour.The keyword may return to its originally-displayed state after one hour,and may be equally displayed as the other contents. A keyword 4805(corresponding to the keyword 4711 in FIG. 47) may be differently markedfrom other contents in FIG. 47, but after one hour, the keyword 4805returns to its originally-displayed state and thus is equally displayedas the other contents. On the other hand, keywords 4817 and 4819 thatare newly added, compared to FIG. 47, may be differently marked from theother contents.

Referring to FIG. 49, the electronic device 100 may time-sequentiallydisplay, on a display 4901, messages received from another electronicdevice, and messages transmitted to the other electronic device. Areference numeral 4903 of FIG. 49 indicates a virtual screen that showsa magnified and extended view of messages displayed on the display 4901.Only a portion of the virtual screen 4903 is displayed on the display4901. That is, when the electronic device 100 time-sequentially displaysthe messages, the electronic device 100 may display apreviously-displayed message on an upper area of the display 4901, andthe time-sequentially displayed messages may be shown on the display4901 while being scrolled.

While the electronic device 100 continuously monitors content ofconversation between users, the electronic device 100 may separatelyindicate or may separately mark a keyword included in the messages onthe display 4901, and may maintain the indication or the mark for apreset time period. The electronic device 100 may extract at least onekeyword by using one or more keywords included in the messages, and maydetermine a query of a search service, based on the extracted at leastone keyword and context information of the user. The electronic device100 may extract the at least one keyword by performing a naturallanguage analysis on contents of the messages. The electronic device 100may extract a keyword by using a previous message and a current messagedisplayed on the display 4901. Also, the electronic device 100 may usethe extracted keyword as a query of the search service, or may deduceanother keyword from the extracted keyword and may use the deducedkeyword as a query of the search service.

The display 4901 may differently mark the keyword to distinguish thekeyword different from other contents of the displayed messages, whereinthe keyword is used in determining the query.

On the virtual screen 4903 of FIG. 49, keywords 4911, 4913, 4915, 4917,and 4919 of communication messages are differently marked from othercontents of the communication messages. The keyword may be differentlymarked from other contents for a preset time period, e.g., for one hour.The keyword may return to its originally-displayed state after one hour,and may be equally displayed as the other contents.

The electronic device 100 may determine the query by using the extractedor deduced keyword and the context information of the user, and mayperform a search operation. The context information of the user mayinclude location information of the user, communication historyinformation of the user, and search log information of the user. Also,the context information of the user may include, but is not limited to,at least one of peripheral environment information of the electronicdevice 100 of the user, state information of the electronic device 100of the user, state information of the user, and schedule information ofthe user.

The peripheral environment information of the electronic device 100indicates information about an environment within a preset radius fromthe electronic device 100, and may include, but is not limited to,weather information, temperature information, humidity information,illumination information, noise information, and sound information.

The state information of the electronic device 100 may include, but isnot limited to, information about a mode of the electronic device 100(e.g., a sound mode, a vibration mode, a silent mode, a power-savingmode, a blocking mode, a multiwindows mode, an automatic rotation mode,etc.), location information of the electronic device 100, timeinformation of the electronic device 100, communication moduleactivation information (e.g., Wi-Fi ON/Bluetooth OFF/GPS ON/NFC ON,etc.), network access state information of the electronic device 100,and information (e.g. ID information of the application, a type of theapplication, a use time of the application, a use period of theapplication, etc.) about at least one application executed in theelectronic device 100.

The state information of the user may indicate information aboutmovement of the user, a life pattern of the user, or the like, and mayinclude, but is not limited to, information about a walking state of theuser, an exercising state of the user, a driving state of the user, asleeping state of the user, and a mood of the user.

When the query is determined, the electronic device 100 may perform thesearch operation without a user input, and may temporally or permanentlystore a search result in a memory of the electronic device 100. Thesearch result may be displayed on the display 4901 according to arequest by the user, or may be displayed on a preset area of the display4901, regardless of a request by the user.

The electronic device 100 may extract the keywords 4911, 4913, 4915,4917, and 4919, may perform search operations by using the extractedkeywords 4911, 4913, 4915, 4917, and 4919 and the context information ofthe user, and may temporally or permanently store search results 4921,4923, 4925, 4927, and 4929 in the memory. For example, the electronicdevice 100 may perform the search operation, based on the extractedkeyword 4911 and the location information of the user, and may store thesearch result 4921 in the memory. Here, the search result 4921 may notbe displayed on the display 4901. Alternatively, the search result 4921may be displayed on the display 4901, according to a user input.Alternatively, the search result 4921 may be displayed on a preset areaof the display 4901, regardless of a user input.

The electronic device 100 may perform the search operation, based on theextracted keyword 4911 and the location information of the user, and maystore the search result 4921 in the memory. Here, the search result 4921may not be displayed on the display 4901. Alternatively, the searchresult 4921 may be displayed on the display 4901, according to a requestby the user. Alternatively, the search result 4921 may be displayed on apreset area of the display 4901, regardless of a request by the user.

The electronic device 100 may perform the search operation, based on theextracted keyword 4913, the location information of the user, and thetime information, and may store the search result 4923 in the memory.The search result 4923 may be filtered, in consideration of a locationclose to a current location of the user, and a current time. Here, thesearch result 4923 may not be displayed on the display 4901.Alternatively, the search result 4923 may be displayed on the display4901, according to a request by the user. Alternatively, the searchresult 4923 may be displayed on a preset area of the display 4901,regardless of a request by the user.

The electronic device 100 may perform the search operations, based onthe extracted keywords 4915, 4917, and 4919, a location of the user, andhistory information of the user, and may store the search results 4925,4927, and 4929 in the memory. The search results 4925, 4927, and 4929may be filtered, in consideration of a location close to a currentlocation of the user, and the history information (e.g., a frequentlyvisited restaurant, a shopping mall, a subway station, etc.). Here, thesearch results 4925, 4927, and 4929 may not be displayed on the display4901. Alternatively, the search results 4925, 4927, and 4929 may bedisplayed on the display 4901, according to a request by the user.Alternatively, the search results 4925, 4927, and 4929 may be displayedon a preset area of the display 4901, regardless of a request by theuser.

Referring to FIG. 50, the electronic device 100 may time-sequentiallydisplay, on a display 5001, messages received from another electronicdevice, and messages transmitted to the other electronic device. Also,the electronic device 100 may display a search result 5005 on thedisplay 5001 while the messages are displayed.

When the electronic device 100 time-sequentially displays the messages,the electronic device 100 may display the messages by scrolling upward apreviously-displayed message.

While the electronic device 100 continuously monitors content ofconversation between users, the electronic device 100 may separatelyindicate or may separately mark a keyword included in the messages, andmay maintain the indication or the mark for a preset time period. Theelectronic device 100 may extract at least one keyword 5003 by using themessages, and may determine a query of a search service, based on thekeyword 5003 and context information of the user. The contextinformation of the user may include location information of the user,communication history information of the user, and search loginformation of the user.

The electronic device 100 may extract the keyword 5003 by performing anatural language analysis on contents of the messages. The electronicdevice 100 may extract the keyword 5003 by using a previous message anda current message displayed on the display 5001.

The display 5001 may differently mark the keyword 5003 to distinguishthe keyword 5003 from other contents of the displayed messages, whereinthe keyword 5003 is used in determining the query.

The keyword 5003 may be differently marked from the other contents for apreset time period, e.g., for one hour. The keyword 5003 may return toits originally-displayed state after one hour, and may be equallydisplayed as the other contents.

The electronic device 100 may determine the query by using the keyword5003 and context information of the user, and may perform a searchoperation. The context information of the user may include locationinformation of the user, communication history information of the user,search log information of the user, and sensing information of theelectronic device 100.

When the query is determined, the electronic device 100 may perform thesearch operation without a user input, and may temporally or permanentlystore the search result in the memory. The search result may bedisplayed on a preset area 5005 of the display 5001 according to arequest by the user, or may be displayed on the preset area 5005 of thedisplay 5001, regardless of a request by the user.

The search result may include distances from a current location of theuser, and recommendation rankings.

The electronic device 100 may receive a user input of selecting akeyword, may perform a search operation, and may display a search resulton the preset area 5005 of the display 5001.

At least one of the components, elements or units represented by a blockas illustrated in the above diagrams may be embodied as various numbersof hardware, software and/or firmware structures that execute respectivefunctions described above, according to an exemplary embodiment. Forexample, at least one of these components, elements or units may use adirect circuit structure, such as a memory, processing, logic, a look-uptable, etc. that may execute the respective functions through controlsof one or more microprocessors or other control apparatuses. Also, atleast one of these components, elements or units may be specificallyembodied by a module, a program, or a part of code, which contains oneor more executable instructions for performing specified logicfunctions. Also, at least one of these components, elements or units mayfurther include a processor such as a central processing unit (CPU) thatperforms the respective functions, a microprocessor, or the like.Functional aspects of the above exemplary embodiments may be implementedin algorithms that are executed by one or more processors and stored ina computer readable recording medium. Furthermore, the components,elements or units represented by a block or processing steps may employany number of related art techniques for electronics configuration,signal processing and/or control, data processing and the like.

It should be understood that exemplary embodiments described hereinshould be considered in a descriptive sense only and not for purposes oflimitation. Descriptions of features or aspects within each exemplaryembodiment should typically be considered as available for other similarfeatures or aspects in other exemplary embodiments.

While one or more exemplary embodiments have been described withreference to the figures, it will be understood by those of ordinaryskill in the art that various changes in form and details may be madetherein without departing from the spirit and scope as defined by thefollowing claims.

What is claimed is:
 1. An electronic apparatus comprising: a display; amemory storing one or more instructions; and a processor configured toexecute the one or more instructions to: display a first window of afirst application, the first window displaying one or more textualmessages exchanged between the electronic apparatus and an externalelectronic apparatus, select a keyword from the one or more textualmessages, display one or more recommended applications regarding theselected keyword, receive a user input to select a recommendedapplication from the one or more recommended applications, and executethe selected recommended application.
 2. The electronic apparatus ofclaim 1, wherein the processor is further configured to execute the oneor more instructions to: identify a category of the selected keyword,and determine the one or more recommended application among applicationsof the electronic apparatus according to the identified category of theselected keyword.
 3. The electronic apparatus of claim 1, wherein theprocessor is further configured to execute the one or more instructionsto display a list indicating the one or more recommended applications.4. The electronic apparatus of claim 2, wherein the processor is furtherconfigured to execute the one or more instructions to determine the oneor more recommended applications which are corresponding to theidentified category of the selected keyword.
 5. The electronic apparatusof claim 1, wherein the processor is further configured to execute theone or more instructions to select the keyword based on an exchangehistory of the one or more textual messages exchanged between theelectronic apparatus and an external electronic apparatus.
 6. Theelectronic apparatus of claim 1, wherein the processor is furtherconfigured to execute the one or more instructions to select the keywordbased on at least one of a user's memo, a user's schedule informationand a user's search history.
 7. The electronic apparatus of claim 1,wherein the processor is further configured to execute the one or moreinstructions to select the keyword based on a user's selection.
 8. Theelectronic apparatus of claim 1, wherein the processor is furtherconfigured to execute the one or more instructions to display a secondwindow of the selected recommended application together with the firstwindow of the first application.
 9. The electronic apparatus of claim 1,wherein an information providing method of an information to be providedfrom the selected recommended application is determined according to atype of the electronic apparatus.
 10. The electronic apparatus of claim1, wherein the processor is further configured to: edit the selectedkeyword, and input the edited keyword into the selected recommendationapplication.
 11. A method of providing information in an electronicapparatus, the method comprising: displaying a first window of a firstapplication, the first window displaying one or more textual messagesexchanged between the electronic apparatus and an external electronicapparatus; selecting a keyword from the one or more textual messages;displaying one or more recommended applications regarding the selectedkeyword; receiving a user input to select a recommended application fromthe one or more recommended applications; and executing the selectedrecommended application.
 12. The electronic apparatus of claim 11,wherein the method further comprises: identifying a category of theselected keyword; and determining the one or more recommendedapplication among applications of the electronic apparatus according tothe identified category of the selected keyword.
 13. The electronicapparatus of claim 11, wherein the displaying the one or morerecommended applications further comprises: displaying a list indicatingthe one or more recommended applications.
 14. The electronic apparatusof claim 12, wherein the method further comprises: determining the oneor more recommended applications which are corresponding to theidentified category of the selected keyword.
 15. The electronicapparatus of claim 11, wherein the selecting the keyword furthercomprises: selecting the keyword based on an exchange history of the oneor more textual messages exchanged between the electronic apparatus andan external electronic apparatus.
 16. The electronic apparatus of claim11, wherein the selecting the keyword further comprises: selecting thekeyword based on at least one of a user's memo, a user's scheduleinformation and a user's search history.
 17. The electronic apparatus ofclaim 11, wherein the selecting the keyword further comprises: selectingthe keyword based on a user's selection.
 18. The electronic apparatus ofclaim 11, wherein the method further comprises: displaying a secondwindow of the selected recommended application together with the firstwindow of the first application.
 19. The electronic apparatus of claim11, wherein an information providing method of an information to beprovided from the selected recommended application is determinedaccording to a type of the electronic apparatus.
 20. At least onenon-transitory computer-readable recording medium for storing a computerprogram, which, when executed by a computer, causes the computer toexecute the method of claim 11.